Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
15 recipes · page 1 of 2
Community builds
Ampere (12)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Production 2x CMP 170HX recipe for Qwen3.8-Flash-Next: zankich INT4PLE repackage of halt95 Merlin W4A16 (94.4 GiB, 51B-param PLE table repacked INT4) on stock vLLM 0.30.0 + a 6-lay
Recipe by nguyenthimy2022kg-alt. Community patched vLLM run of the NVIDIA Qwen3.8-Flash-Next NVFP4 checkpoint on 2x unlocked CMP 170HX (SM80, ~63.4 GiB usable each) with TEP2 tenso
Tensor-parallel successor to Flun's 2x CMP 170HX layer-split recipe: ExLlamaV3 TP2 over both 64 GB cards with a 655,360-token Q8 KV pool (392,960-token request cap), DFlash2 K7 spe
First ExLlamaV3 serving recipe on 2x CMP 170HX: full-VRAM HBM-resident GLM-5.3-Flash EXL3 3.05bpw (~116.6 GiB across 2x64GB, no CPU/SSD tier) with DFlash2 K7 speculative decoding —
Qwen3.8-Flash-Next (125B MoE) INT4-expert W4A16 on two CMP 170HX cards, full 262,144-token context: 241/350/198 tok/s one user (structured/code/prose), 495 tok/s editing code, 895/
Recipe by AlexBrandes. Qwen3.8-Flash-Next (Intel/Qwen3.8-Flash-Next-W4A16-AutoRound @ 4c67bf68) on two NVIDIA CMP 170HX cards in PIPELINE parallel (PP2, layer split 26/22) with a v
Two-card Ampere pipeline recipe for Qwen3.8-Flash-Next-Uncensored (512-expert MoE), AWQ W4A16 g32: PP2 (CMP 170HX has no P2P), MTP 5-token drafting with an int4 draft-only lm_head,
First EXL3 row for Qwen3.8-Flash-Next and first Morrowmake QFN lane: turboderp/Qwen3.8-Flash-Next-exl3 (EXL3 5.05 bpw) on two CMP 170HX cards (TP2, whole routed experts per card, P
2x CMP 170HX (GA100 sm_80, 64 GB/card, cmpunlocker unlock) recipe for Qwen3.8-Flash-Next from bozocentral's writeup (2026-10-07; page-reference: artifact IS the URL). Minachist INT
Showing 12 of 15 recipes on this page — filter or paginate for the rest.