Qwen2.5 0.5B

model page · concurrency sweep 1–24 agents

Qwen2.5 0.5B

server model id: qwen2.5-0.5b-instruct
non-thinkingKV F16ctx 32k0.5B params

The throughput monster — fastest solo of all (354 tok/s) and fastest visible-content rate on record (944 tok/s at 20 agents) on full-precision F16 KV.

Key findings

  • Fastest absolute throughput of any model tested: 944 tok/s at 20 agents (and 936 at 16, 871 at 24) — well ahead of the previous best (falcon3 839, qwen-deepseek 829, minicpm 983* measured all-token). The tiny 0.5b model plus F16 KV makes it a throughput monster: single-agent is 354 tok/s (also the fastest solo of the whole set).
  • Solo run finished early on budget: concurrency 1 hit the 12k budget in 33.9s, concurrency 2 in 45.6s — the only runs that didn't use the full 60s, because the model out-runs the budget.
  • Low relative scaling (100% → 267%) — but that's a good thing here: the model is already near capacity at 2 agents (527 tok/s), so concurrency adds little relative headroom. The mid-range (3–14) grows slowly, then a jump at 15–20.
  • Excellent latency: TTFT 55ms solo, mostly 2–14s under load with no pathological spikes.
  • All visible content, 0 reasoning tokens.
  • Errors: 0 across all 24 runs.

Sweep — total concurrency 1–24

agentswall (s)contentreasontotalcomb tok/scomb allper-agentper-agent allscale %TTFT meanTTFT maxoktimeouterr
133.912,000012,000354354354.8354.8100%55ms55ms100
245.624,000024,000527527264.1268.0149%76ms84ms200
360.030,077030,077501501472.2489.6142%9.7s9.7s030
460.032,129032,129535535166.9178.3151%8.9s8.9s040
560.032,476032,476541541161.4169.9153%6.4s6.4s050
660.132,443032,443540540134.0139.0153%8.8s8.8s060
760.032,882032,882548548199.1201.8155%11.0s11.0s070
860.033,382033,38255655685.989.2157%696ms723ms080
960.134,893034,893581581146.1148.6164%9.9s10.0s090
1060.136,567036,567609609117.4119.0172%10.4s10.4s0100
1160.037,939037,939632632100.9105.3179%10.7s10.7s0110
1260.138,531038,53164264273.676.4181%5.7s5.8s0120
1360.137,386037,38662362373.574.5176%14.4s14.5s0130
1460.138,049038,04963463475.976.8179%12.6s12.6s0140
1560.143,862043,86273073068.170.7206%13.9s14.0s0150
1660.156,214056,21493693669.971.9264%2.0s2.0s0160
1760.154,694054,69491191181.985.0257%5.2s5.2s0170
1860.151,477051,47785785779.282.1242%9.9s9.9s0180
1960.151,674051,67486086060.261.9243%9.4s9.5s0190
20 peak60.156,706056,70694494471.774.2267%4.4s4.5s0200
2160.154,171054,17190290270.072.4255%10.1s10.1s0210
2260.151,814051,81486386364.667.4244%8.2s8.3s0220
2360.152,938052,93888188174.077.3249%11.0s11.1s0230
2460.152,334052,33487187173.477.0246%12.3s12.3s0240

* burst artifact — agents queued ~11–16s then generated in a short burst; the per-agent timer inflates the number. Trust combined throughput and TTFT columns. Peak row = highest combined (all-token) throughput.

Charts

Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Scaling efficiency relative to the single-agent baseline (100%).
Scaling efficiency relative to the single-agent baseline (100%).
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Total tokens per 60s run — all tokens vs visible content only.
Total tokens per 60s run — all tokens vs visible content only.
How each run of N agents finished: completed, timed out (60s budget), or errored.
How each run of N agents finished: completed, timed out (60s budget), or errored.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.

Raw data & downloads