Llama 3.2 1B Mini-Agent

model page · concurrency sweep 1–24 agents

Llama 3.2 1B Mini-Agent

server model id: llama-3.2-1b-mini-agent
non-thinkingKV F16ctx 32k1B params

The first agentic-tuned model tested — clean 196 tok/s solo, plateau of ~600–650 tok/s from 13 agents on.

Key findings

  • Throughput: combined tok/s rises from 196 (1 agent) to a plateau of ~600–650 tok/s at 13+ agents; peak is 652 tok/s at 24 agents (3.3x single-agent). Adding agents past ~13–16 buys almost nothing.
  • Per-agent speed: erodes from 209 tok/s alone to 38–55 tok/s at 17–24 agents (19–27% of single-agent throughput) due to slot contention.
  • TTFT: highly variable (188ms to ~10.4s) across runs — reflects LM Studio batching/prefill under load, not a clean linear queue.
  • Errors: 0 across all 24 runs.
  • Caveat: at higher concurrency some agents end early via an "empty-turn" stop (model returns a turn with no content while competing for a slot); these are counted as ok with roughly 1.1–1.9k tokens rather than reaching the 12k budget.

Sweep — total concurrency 1–24

agentswall (s)contentreasontotalcomb tok/scomb allper-agentper-agent allscale %TTFT meanTTFT maxoktimeouterr
160.011,741011,741196196209.3209.3100%880ms880ms010
260.018,908018,908315315171.4171.4161%188ms194ms020
360.020,642020,642344344161.3161.3176%10.4s10.4s030
460.025,816025,816430430169.2169.2219%8.8s8.8s040
560.028,081028,081468468126.1126.1239%9.4s9.4s050
660.027,983027,983466466107.8107.8238%10.4s10.4s060
760.030,879030,87951451481.781.7262%3.0s3.0s070
860.032,819032,81954754781.381.3279%782ms819ms170
960.130,729030,72951251270.170.1261%555ms578ms270
1060.033,683033,68356156173.373.3286%281ms295ms370
1160.036,650036,65061061080.080.0311%570ms594ms560
1260.136,701036,70161161171.071.0312%1.0s1.0s480
1360.138,681038,68164464472.472.4329%1.5s1.5s670
1460.136,525036,52560860864.064.0310%1.5s1.6s4100
1560.137,062037,06261761759.959.9315%854ms908ms5100
1660.138,106038,10663463461.261.2323%2.2s2.2s6100
1760.133,519033,51955855852.252.2285%258ms308ms7100
1860.033,750033,75056256243.043.0287%1.1s1.2s6120
1960.138,547038,54764264254.854.8328%716ms794ms7120
2060.037,080037,08061861847.447.4315%638ms751ms8120
2160.137,177037,17761961946.046.0316%591ms650ms10110
2260.136,271036,27160460445.545.5308%676ms761ms9130
2360.035,518035,51859159138.138.1302%1.7s1.9s9140
24 peak60.139,161039,16165265245.045.0333%674ms791ms12120

* burst artifact — agents queued ~11–16s then generated in a short burst; the per-agent timer inflates the number. Trust combined throughput and TTFT columns. Peak row = highest combined (all-token) throughput.

Charts

Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Whole sweep at a glance — combined throughput, scaling efficiency, per-agent throughput and outcomes across every concurrency level.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Combined throughput (all tokens) vs concurrent agents. Annotated peak with the concurrency level where it occurred.
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Per-agent throughput erosion as slots contend. Red stars mark burst-timing artifacts (queued ≥ 10s then generated in a burst — inflated values, not real throughput).
Scaling efficiency relative to the single-agent baseline (100%).
Scaling efficiency relative to the single-agent baseline (100%).
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Time to first token — mean (solid) and max (dashed). Log scale. Gaps mean no visible content ever arrived within the 60s window.
Total tokens per 60s run — all tokens vs visible content only.
Total tokens per 60s run — all tokens vs visible content only.
How each run of N agents finished: completed, timed out (60s budget), or errored.
How each run of N agents finished: completed, timed out (60s budget), or errored.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.
The concurrency trade-off on one chart — combined throughput rises while per-agent speed falls.

Raw data & downloads