GLM-5.3-Flash NVFP4 (nvidia checkpoint) served by Enntity's Atlas engine with tensor+expert parallelism across two DGX Sparks over one ConnectX-7 cable, DFlash2 speculative decodin
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
20 recipes · page 1 of 2
Community builds
Blackwell (4)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3
DeepSeek-V4.1-Flash sglang NVIDIA DGX Spark 128GB ×4 (Blackwell) 89.7 tok/s decode · 5398 tok/s prefill
Recipe by ttlogan. Production 2-node GB10 (ASUS GX10 / DGX Spark) cluster serve of jpezzulli/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-ModelOpt-NVFP4 (OrcaRouter uncensored weights
Qwen3.8-Flash-Next sglang NVIDIA DGX Spark 128GB ×2 (Blackwell) 40.3 tok/s decode · 2936.7 tok/s prefill
Ampere (8)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Production 2x CMP 170HX recipe for Qwen3.8-Flash-Next: zankich INT4PLE repackage of halt95 Merlin W4A16 (94.4 GiB, 51B-param PLE table repacked INT4) on stock vLLM 0.30.0 + a 6-lay
Recipe by nguyenthimy2022kg-alt. Community patched vLLM run of the NVIDIA Qwen3.8-Flash-Next NVFP4 checkpoint on 2x unlocked CMP 170HX (SM80, ~63.4 GiB usable each) with TEP2 tenso
Tensor-parallel successor to Flun's 2x CMP 170HX layer-split recipe: ExLlamaV3 TP2 over both 64 GB cards with a 655,360-token Q8 KV pool (392,960-token request cap), DFlash2 K7 spe
First ExLlamaV3 serving recipe on 2x CMP 170HX: full-VRAM HBM-resident GLM-5.3-Flash EXL3 3.05bpw (~116.6 GiB across 2x64GB, no CPU/SSD tier) with DFlash2 K7 speculative decoding —
Qwen3.8-Flash-Next (125B MoE) INT4-expert W4A16 on two CMP 170HX cards, full 262,144-token context: 241/350/198 tok/s one user (structured/code/prose), 495 tok/s editing code, 895/
Showing 12 of 20 recipes on this page — filter or paginate for the rest.