DeepSeek-V4.1-Flash-1017 (748B MoE) on four CMP 170HX cards (sm_80): Morrowmake's release 1.0.0 recipe serves a hybrid checkpoint - routed experts from LibertAI's EXL3 2.9-bpw expo
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
30 recipes · page 1 of 3
Ampere (2)
PP4 (pipeline-parallel 4, TP1) as-run on a 4x CMP 170HX rig 2026-09-28 at --gpu-memory-utilization 0.93 after 0.95 OOM-killed warmup (a 72 MiB KDA prefill workspace did not fit in
Community builds
Blackwell (1)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
GLM-5.3-Flash (321B MoE, ~18B active) turboderp EXL3 2.05bpw on two RTX 5090 + 64 GB DDR5: TabbyAPI on ExLlamaV3 with CPU-offloaded routed experts (96/layer GPU0 layers 3-22+MTP, 1
Ampere (9)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Recipe by g4bwy. Official Qwen/Qwen3.8-Flash-Next-FP8 on 4x unlocked CMP 170HX (GA100 sm_80, 74 SMs, 64 GB) with PP=4 (partition 13,12,12,11) and MTP-4 speculative decoding on pinn
DeepSeek-V4.1-Flash-1017 (748B MoE) on 4x CMP 170HX (sm_80, 64 GB each, PCIe Gen2 x16), TP4, as a fidelity-first variant of the GNmac-lab recipe 1.2.0 @ 2ad5741ab1 with four change
Qwen3.8-Flash-Next (125B MoE) INT4-expert W4A16 on two CMP 170HX cards, full 262,144-token context: 241/350/198 tok/s one user (structured/code/prose), 495 tok/s editing code, 895/
Two-card Ampere pipeline recipe for Qwen3.8-Flash-Next-Uncensored (512-expert MoE) in AWQ W4A16 (leoncca g32 build @ fa56146): PP2 with manual layer partition 25,23 (no P2P on CMP
Two-card Ampere pipeline recipe for Qwen3.8-Flash-Next-Uncensored (512-expert MoE), AWQ W4A16 g32: PP2 (CMP 170HX has no P2P), MTP 5-token drafting with an int4 draft-only lm_head,
DeepSeek-V4.1-Flash (ORIGINAL deepseek-ai/DeepSeek-V4.1-Flash checkpoint, quantized by Mia-AiLab as EXL3 3.02bpw — NOT the 2026-10-17 '1017' re-release, which has its own sibling r
DeepSeek-V4.1-Flash-1017 (the 2026-10-17 re-release checkpoint, NOT the original DeepSeek-V4.1-Flash) as a hybrid quant on the TC-MoE port: official deepseek-ai/DeepSeek-V4.1-Flash
Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, D
Showing 12 of 30 recipes on this page — filter or paginate for the rest.