Benchmarks
We don’t run our own benchmark harness. Speeds on this site are reported — they come from recipe submitters and imported run files. The community benchmarks local LLM serving with llm-inference-bench; recipe-level speeds stay on each recipe’ detail page.
GLM-5.3-Flash (6 runs)
| Run | Headline | Model | Layout | Hardware | Date | Tier | |
|---|---|---|---|---|---|---|---|
| llm-inference-bench 0.8.0 | 41.5 tok/s | GLM-5.3-Flash | layout: tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | NVIDIA CMP 170HX ×2 (Ampere) | 13 d ago | spot-check | view run |
| llm-inference-bench 0.8.0 | 43.2 tok/s | GLM-5.3-Flash | layout: tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | NVIDIA CMP 170HX ×2 (Ampere) | 13 d ago | spot-check | view run |
| llm-inference-bench 0.8.0 | 41.7 tok/s | GLM-5.3-Flash | layout: tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | NVIDIA CMP 170HX ×2 (Ampere) | 13 d ago | spot-check | view run |
| llm-inference-bench 0.8.0 | — | GLM-5.3-Flash | layout: tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | NVIDIA CMP 170HX ×2 (Ampere) | 13 d ago | spot-check | view run |
| llm-inference-bench 0.8.0 | — | GLM-5.3-Flash | layout: tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | NVIDIA CMP 170HX ×2 (Ampere) | 13 d ago | spot-check | view run |
| llm-inference-bench 0.8.0 | — | GLM-5.3-Flash | layout: tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | NVIDIA CMP 170HX ×2 (Ampere) | 13 d ago | spot-check | view run |
Run files
Newest 50 imported benchmark runs — every row links its full run report. Unattached runs (no recipe) show here too.
| date | tool | model | layout | tier | headline | run |
|---|---|---|---|---|---|---|
| 2026-10-17 | llm-inference-bench 0.8.0 | GLM-5.3-Flash | tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | spot-check | 41.5 tok/s | view run |
| 2026-10-17 | llm-inference-bench 0.8.0 | GLM-5.3-Flash | tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | spot-check | 43.2 tok/s | view run |
| 2026-10-17 | llm-inference-bench 0.8.0 | GLM-5.3-Flash | tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | spot-check | 41.7 tok/s | view run |
| 2026-10-17 | llm-inference-bench 0.8.0 | GLM-5.3-Flash | tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | spot-check | — | view run |
| 2026-10-17 | llm-inference-bench 0.8.0 | GLM-5.3-Flash | tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | spot-check | — | view run |
| 2026-10-17 | llm-inference-bench 0.8.0 | GLM-5.3-Flash | tp ×TP2 on 2x CMP170HX 64GiB, 4 active sequences | spot-check | — | view run |
Submit a benchmark
Run llm-inference-bench or tool-eval-bench against your rig? Post the run file — it gets a permanent run-report page and (when attached) feeds the measured boards.
Benchmark sources
Local Inference Lab
Independent nonprofit — open kernels, tools, benchmarks, and research for AI inference on hardware you control. Its llm-inference-bench is the best benchmarking tool for local LLM serving — it produces the run files behind most community speed numbers, and results get posted in their Discord.
Website · llm-inference-bench · Releases · GitHub org · Discord · Docker launcher · Benchmarks · Hugging Face · X
Artificial Analysis
Independent model benchmarks — source of the AI Intelligence Index values shown on our leaderboard.
Engine docs (vLLM, SGLang) and community links live on the Helpful Links page.