Qwen3.5 0.8B — MTP Enabled

model page · concurrency sweep 1–24 agents

Qwen3.5 0.8B — MTP Enabled

server model id: qwen3.5-0.8b-mtp
thinkingKV F16ctx 32k0.8B params

The cautionary tale: MTP serializes requests — flat ~180 tok/s at any concurrency and zero visible content from 10 agents up.

Key findings

  • No visible output under load: from concurrency ~10 up, agents produce zero content tokens in 60s — the entire budget goes to reasoning_content (all 10k+ tokens). Even the TTFT-to-first-content never arrives.
  • Massive reasoning preamble: even solo, first visible content takes ~22s and only appears after ~3.5k reasoning tokens. Probing a fresh request returned content: '' with 100% reasoning.
  • Flat throughput — no concurrency scaling: combined (all-token) throughput is ~180 tok/s from 2 to 24 agents (202 solo). Total per run ≈ one stream's worth (~10–11k tokens). This is the signature of request serialization — consistent with MTP (multi-token prediction) using a decode path that LM Studio cannot batch, so every request queues on one slot. Concurrency adds only latency (TTFT 20–53s), never throughput.
  • Errors: 0 across all 24 runs — but the model is effectively unusable for the NPC/controller workload as served.

Sweep — total concurrency 1–24

agentswall (s)contentreasontotalcomb tok/scomb allper-agentper-agent allscale %TTFT meanTTFT maxoktimeouterr
1 peak60.08,6233,51312,136144202235.3208.1100%21.8s21.8s010
260.03,9445,7989,742661620.00.080%33.8s39.6s020
360.02,3127,71810,030391670.00.083%36.7s42.7s030
460.03,1476,98010,127521690.00.084%35.8s52.0s040
560.01,0179,44210,459171740.00.086%43.9s45.1s050
660.029810,58610,88451810.00.090%53.8s57.7s060
760.051810,15210,67091780.00.088%40.3s40.3s070
860.017810,73310,91131820.00.090%51.7s51.7s080
960.084010,00410,844141810.00.090%19.4s19.4s090
1060.0010,86110,86101810.00.090%——0100
1160.0010,99410,99401830.00.091%——0110
1260.023310,65810,89141810.00.090%42.7s42.7s0120
1360.0011,05611,05601840.00.091%——0130
1460.0011,01211,01201830.00.091%——0140
1560.0010,86710,86701810.00.090%——0150
1660.0010,97110,97101830.00.091%——0160
1760.0010,84110,84101810.00.090%——0170
1860.0010,94610,94601820.00.090%——0180
1960.0010,77210,77201790.00.089%——0190
2060.0010,36510,36501730.00.086%——0200
2160.0010,94610,94601820.00.090%——0210
2260.0011,31811,31801890.00.094%——0220
2360.0010,91310,91301820.00.090%——0230
2460.0010,31710,31701720.00.085%——0240

* burst artifact — agents queued ~11–16s then generated in a short burst; the per-agent timer inflates the number. Trust combined throughput and TTFT columns. Peak row = highest combined (all-token) throughput.

Charts

Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Scaling efficiency relative to the single-agent baseline (100%).
Scaling efficiency relative to the single-agent baseline (100%).
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Total tokens per 60s run — all tokens vs visible content only.
Total tokens per 60s run — all tokens vs visible content only.
How each run of N agents finished: completed, timed out (60s budget), or errored.
How each run of N agents finished: completed, timed out (60s budget), or errored.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.
Visible content vs hidden reasoning tokens per run, plus the reasoning share (%).
Visible content vs hidden reasoning tokens per run, plus the reasoning share (%).

Raw data & downloads