Three models, one DGX Spark: the one that reads the fewest bytes loses
Three models fit comfortably on a DGX Spark’s 121 GB of unified memory, and they represent three different bets about where performance comes from: a dense 27B at 8 bits, a 35B mixture-of-experts at 16 bits with about 3B active per token, and a 117B mixture-of-experts at 4 bits with about 5.1B active.
Measured on the same machine, same harness, same 32k context, the ranking is not what parameter counts suggest. It is also not what the standard mental model for memory-bound decode suggests, which is the more interesting half.
What is being compared
| Qwen3.6-27B | Qwen3.6-35B-A3B | gpt-oss-120b | |
|---|---|---|---|
| Architecture | Dense | MoE, ~3B active | MoE, ~5.1B active |
| Total parameters | 27B | 35B | 117B |
| Quantization | FP8 | bf16 | MXFP4 |
| Speculative decoding | MTP, 2 tokens | MTP, 2 tokens | none |
| Context | 32,768 | 32,768 | 32,768 |
--gpu-memory-utilization |
0.65 | 0.65 | 0.65 |
All three served by vLLM 0.19.0 on the same GB10, one at a time.
The metric is end-to-end throughput: total tokens generated divided by total wall time. It is the only figure available for all three, because vLLM buffers gpt-oss’s stream and makes token-level timings meaningless there. It includes prefill, so it is a floor on decode rather than a substitute for it.
Results
chat_short125 prompt tokens · median of 5 runs| Machine | Model | tok/s | |
|---|---|---|---|
NVIDIA DGX Spark (GB10) NVIDIA GB10 | Qwen/Qwen3.6-35B-A3B bf16 · vllm · mtp spec · 32k ctx | 50.2 ±1.2 | |
NVIDIA DGX Spark (GB10) NVIDIA GB10 | openai/gpt-oss-120b MXFP4 · vllm | 37.2 ±0.2 | |
NVIDIA DGX Spark (GB10) NVIDIA GB10 | Qwen/Qwen3.6-27B FP8 · vllm · mtp spec · 32k ctx | 17.4 ±0.2 |
| Scenario | Prompt | Output | 27B | 35B-A3B | gpt-oss-120b |
|---|---|---|---|---|---|
chat_short |
~150 | 256 | 17.4 | 50.2 | 37.2 |
code_generation |
~400 | 1024 | 16.6 | 50.4 | 37.0 |
chat_long_context |
~5,400 | 256 | 13.6 | 37.2 | 30.7 |
summarization |
~10,700 | 128 | 8.9 | 23.0 | 19.5 |
The 35B-A3B wins every scenario. gpt-oss is second everywhere, and roughly 2.1× faster than the 27B throughout — a model with four times the parameters, running twice as fast, consistently.
The 27B result needs no explanation
Decode on this hardware is memory-bandwidth-bound. Producing one token means reading the active weights out of memory, and the GB10 has roughly 273 GB/s to work with. So count bytes:
| Model | Active params | Bytes each | Read per token |
|---|---|---|---|
| Qwen3.6-27B | 27B (all of them) | 1 (FP8) | ~27 GB |
| Qwen3.6-35B-A3B | ~3B | 2 (bf16) | ~6 GB |
| gpt-oss-120b | ~5.1B | 0.5 (MXFP4) | ~2.6 GB |
A dense model pays for every parameter on every token. The 27B moves ten times the data gpt-oss does, and measures roughly half the speed. That is the whole story there, and it is the reason “how many parameters” is a poor proxy for “how fast”: what matters is how many of them you touch, and how wide each one is.
The part that does not work
Read that table again. gpt-oss reads the fewest bytes per token of the three — 2.6 GB against the 35B’s 6 GB. On a purely bandwidth-bound view it should be roughly 2.3× faster than the 35B.
It is 26% slower.
So the model that explains the 27B cleanly gives the wrong answer for the two MoEs. Something is adding time that bytes-per-token does not account for.
The leading candidate is in the configuration table: both Qwen models run MTP speculative decoding, and gpt-oss does not. Speculative decoding drafts several tokens per forward pass and has the full model verify them; when drafts are accepted, you get multiple tokens for one weight read. It is precisely a way to beat the bandwidth bound. Measurements on this machine put the 35B’s draft acceptance around 91%.
Back out a 1.5–2× speculative speedup and the 35B lands somewhere near 25–33 tok/s — below gpt-oss’s 37, which is the ordering bytes-per-token predicts.
I want to be clear that this is a hypothesis, not a result. It fits, and
the mechanism is right, but I have not tested it. The experiment is
straightforward: serve the 35B without --speculative-config and measure
again. If it drops into the low thirties, the explanation holds. If it barely
moves, something else is going on and I would want to know what. That is the
next post.
What is safe to say now is narrower and still useful: a decoding technique can outweigh a 2.3× advantage in bytes moved per token. If you are choosing between models on throughput, the serving configuration matters as much as the architecture, and it is the thing most comparisons leave out.
Prefill, measured separately
Prefill needed its own method, because deriving it from time-to-first-token requires a server that streams token by token, which gpt-oss does not. Holding output length fixed and varying prompt length isolates it from total wall time instead — the slope of that line is the prefill rate.
| Model | Prefill |
|---|---|
| Qwen3.6-27B | ~1,720 tok/s |
| Qwen3.6-35B-A3B | ~3,900 tok/s |
| gpt-oss-120b | ~2,500 tok/s |
Same ordering as decode, which is why the ranking holds across every prompt length rather than crossing over somewhere. Nobody wins at long context by having unusually fast prefill.
That method agreed with the streaming measurement to within 2% on the 27B and 6% on the 35B, which is the only reason I trust it on gpt-oss, where there is nothing to check it against.
Cold starts
None of these are always-on. Time from “a client asks” to “the endpoint answers”, measured with an empty GPU:
| Model | Cold start |
|---|---|
| Qwen3.6-35B-A3B | 435–460 s |
| Qwen3.6-27B | 505–555 s |
| gpt-oss-120b | 570 s |
Seven to nine and a half minutes. Note the 27B — 27 GB of FP8 weights — loads slower than the 35B’s 70 GB of bf16. Weight size is not the bottleneck; engine initialisation is. I still cannot account for that one.
What I would run
The 35B-A3B, on this hardware, for general work. It is fastest at every prompt length by a comfortable margin, and it loads quickest.
gpt-oss-120b if you want the largest model that fits and can accept 26% less throughput. It is a 117B model running at 37 tok/s on a desk-side box, which is a genuinely surprising thing to be able to say.
The 27B only if you have measured it winning on your specific task. It is the slowest of the three by a factor of two, and the dense architecture is why.
What this does not tell you
Quality. These are throughput numbers and nothing else. I have MMLU-Pro scores for the two Qwen models — 82.1% and 81.4%, statistically indistinguishable — and none at all for gpt-oss, so a three-way quality comparison would be two-thirds of a table. That measurement is running now and is the next post.
Until then, the honest reading of this article is “which of these is fastest”, not “which of these you should run”. Those are different questions, and the first one only decides the second when the answers to the second are equal.
Concurrency. Single-stream throughout: one request at a time. Batching changes the picture and is not measured here.
Methodology
Five measured repetitions per scenario after a discarded warmup, greedy decoding, medians reported, run under a coordination gate that refuses to measure while another project is using the GPU.
Each repetition uses a unique prompt prefix. That detail is not incidental: vLLM enables automatic prefix caching, and an earlier version of these measurements sent an identical prompt every time. Every recorded repetition was a cache hit, and the long-prompt numbers for gpt-oss came out up to 81% too high before I caught it. That correction is documented in the gpt-oss post. The Qwen models were unaffected — speculative decoding disables prefix caching, which is a coincidence I only understood after measuring both ways.
All three runs here were taken after that fix, with the same harness on the same machine on the same day.
Source data: b43438259dc696f13c64aa0699ee050fe7f2