Measured benchmarks for running large language models on hardware you own. No vendor benchmarks, no reprinted spec sheets — just numbers measured on machines sitting in my office.
Running gpt-oss-120b on a DGX Spark: four things that will trip you up
A 117B-parameter model runs at ~37 tok/s on a desk-side box, alongside three other models. Getting there involves a 133 GB download that should be 57 GB, a mandatory backend flag, and two separate ways the benchmark lied to me — one of which I published before catching.
Three models, one DGX Spark: the one that reads the fewest bytes loses
Qwen3.6-27B, Qwen3.6-35B-A3B and gpt-oss-120b measured identically on one GB10. The 117B model is twice as fast as the 27B, the bytes-per-token model predicts it should win outright, and it does not.
lm-eval scored a good model at 13% on MMLU-Pro. The model was fine.
A stop sequence in lm-evaluation-harness fires inside a reasoning model's own chain-of-thought, truncating every answer before it arrives. One flag moves the score from 13.3% to 86.7%.
A 35B MoE runs 3× faster than a 27B dense model on the DGX Spark
Measured speed and quality for Qwen3.6-35B-A3B and Qwen3.6-27B on a GB10 — decode, prefill, time-to-first-token, cold starts and MMLU-Pro. The MoE wins on speed by 3x, and the dense model does not visibly win on quality.