Qwen2.5 0.5B ×2 (parallel)

model page · concurrency sweep 2–48 agents · 2 parallel instances

Qwen2.5 0.5B ×2 (parallel)

server model id: qwen2.5-0.5b-instruct / qwen2.5-0.5b-instruct-2
2× parallel instancesnon-thinkingKV F16ctx 32k0.5B params

The same 0.5B model with two parallel LM Studio instances sharing the GPU: 1020 tok/s inside the 24-agent window, up to 1815 tok/s at 38 total agents (5.1x).

Key findings

  • Two instances roughly double throughput: combined output reaches 1815 tok/s at 38 total agents (5.1x the 2-agent baseline) and ~1400–1610 from 40–48 — roughly 2x the ~900–944 tok/s ceiling of the single-instance run. Running two API-handler slots in parallel works.
  • Steeper scaling than single-instance: single-instance plateaued at ~267% (relative to its solo); the dual run keeps climbing to 506% at 38 — the extra slot gives real headroom.
  • Solo-baseline caveat: the 2-agent row (1 per instance) is 359 tok/s, similar to the single-instance 354 — each slot independently sustains its solo speed, and combining them adds up.
  • Noisy mid-range, then a jump at 30–34: throughput swings 701–1020 between 8–30, then jumps to ~1446–1815 at 32–38 — LM Studio's per-slot batching transitions.
  • TTFT: 111ms–27s, erratic like the single-instance run; no pathological outliers.
  • All visible content, 0 reasoning tokens.
  • Errors: 0 across all 24 levels.

Sweep — total concurrency 2–48

agentswall (s)contentreasontotalcomb tok/scomb allper-agentper-agent allscale %TTFT meanTTFT maxoktimeouterr
260.021,558021,558359359205.6205.6100%7.6s11.1s020
460.034,264034,264571571151.1168.8159%111ms129ms040
660.050,226050,226836836583.9599.6233%2.0s2.1s060
860.043,370043,370722722225.1229.8201%16.2s17.3s080
1060.042,075042,075701701131.3135.2195%15.2s15.5s0100
1260.046,594046,594776776113.2116.4216%11.8s13.3s0120
1460.050,885050,885847847131.4133.2236%13.7s14.9s0140
1660.049,695049,69582882888.189.0231%16.2s18.4s0160
1860.053,576053,576892892113.4114.4248%19.4s20.1s0180
2060.055,550055,55092592595.296.8258%19.7s21.0s0200
2260.061,222061,2221,0201,02098.599.5284%21.0s23.1s0220
2460.151,204051,20485385388.589.2238%27.2s28.2s0240
2660.160,695060,6951,0111,01184.985.5282%18.9s21.0s0260
2860.157,712057,71296196185.486.3268%26.1s26.7s0280
3060.164,897064,8971,0811,08176.977.4301%25.0s26.3s0300
3260.186,878086,8781,4461,44671.672.9403%10.5s15.4s0320
3460.195,454095,4541,5891,58975.076.9443%7.8s9.5s0340
3660.185,474085,4741,4231,42353.956.0396%13.5s14.2s0360
38 peak60.1109,0080109,0081,8151,81566.468.5506%3.6s3.9s0380
4060.195,521095,5211,5901,59066.368.9443%13.9s15.9s0400
4260.193,382093,3821,5551,55563.266.4433%13.2s14.5s0420
4460.193,846093,8461,5621,56265.468.6435%13.6s15.2s0440
4660.184,457084,4571,4061,40655.558.7392%19.0s20.1s0460
4860.196,869096,8691,6121,61265.369.3449%13.0s13.8s0480

Parallel run: N agents ran on each of two LM Studio instances simultaneously (total 2×N). * burst artifact — agents queued ~11–16s then generated in a short burst; the per-agent timer inflates the number. Trust combined throughput and TTFT columns. Peak row = highest combined (all-token) throughput.

Charts

Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Scaling efficiency relative to the single-agent baseline (100%).
Scaling efficiency relative to the single-agent baseline (100%).
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Total tokens per 60s run — all tokens vs visible content only.
Total tokens per 60s run — all tokens vs visible content only.
How each run of N agents finished: completed, timed out (60s budget), or errored.
How each run of N agents finished: completed, timed out (60s budget), or errored.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.

Raw data & downloads