The proven 3-card pipeline-parallel recipe on CMP 170HX (sm_80, 64GB unlocked): PP=3 with manual layer partition 15,16,12 at util 0.98, FP8 KV, block-size 256, DSpark block draftin
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
4 recipes · page 1 of 1
Ampere (1)
Community builds
Blackwell (2)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3
Pipeline-parallel TP1xPP2 + DSpark serving of DeepSeek V4.1 Flash (1017 official FP8 weights) across two DGX Station GB300 nodes over 400G CX8 RoCE. PP2 splits the 40 layers 20/20
Ampere (1)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Working PP3 recipe for Qwen3.8-Flash-Next (Qwen4Exp 180B-A6B MoE, Intel INT4 W4A16 AutoRound) on 3x CMP 170HX (SM80, unlocked 64 GiB each): the 95.4 GiB BF16 PLE n-gram table strea