DeepSeek-V4.1-Flash-1017 (748B MoE) on four CMP 170HX cards (sm_80): Morrowmake's release 1.0.2 recipe serves a hybrid checkpoint - routed experts from LibertAI's EXL3 2.9-bpw expo
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
33 recipes · page 1 of 3
Ampere (1)
Community builds
Blackwell (1)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Streams a 483 GiB DeepSeek-V4.1-Flash-1017 pack from NVMe straight into one 32 GB RTX 5090, so a frontier-size MoE runs on a desktop with 32 GB of VRAM and 32 GB of system RAM. Win
DeepSeek-V4.1-Flash-1017 llama.cpp NVIDIA RTX 5090 32GB ×1 (Blackwell) 17.8 tok/s decode · 3356 tok/s prefill
Ampere (10)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Recipe by g4bwy. Official Qwen/Qwen3.8-Flash-Next-FP8 on 4x unlocked CMP 170HX (GA100 sm_80, 74 SMs, 64 GB) with PP=4 (partition 13,12,12,11) and MTP-4 speculative decoding on pinn
DeepSeek-V4.1-Flash (ORIGINAL deepseek-ai/DeepSeek-V4.1-Flash checkpoint, quantized by Mia-AiLab as EXL3 3.02bpw — NOT the 2026-10-17 '1017' re-release, which has its own sibling r
DeepSeek-V4.1-Flash-1017 (the 2026-10-17 re-release checkpoint, NOT the original DeepSeek-V4.1-Flash) as a hybrid quant on the TC-MoE port: official deepseek-ai/DeepSeek-V4.1-Flash
DeepSeek-V4.1-Flash-1017 (748B MoE) on four CMP 170HX cards power-capped at 180 W, with the model's 189 GiB Engram n-gram tables streamed from NVMe on demand. Recipe 1.2.0 (author
Everyday GLM-5.3-Flash setup on 2x CMP 170HX unlocked to 64 GB (PCIe Gen2 x4, 180 W each, 32 GB RAM, 850 W PSU): alexvnesta's ExLlamaV3 server in layer-split mode, MTP head draftin
DeepSeek-V4.1-Flash-1017 (491 GB, no requantization) served on a single 64 GB CMP 170HX. XTier (v0.1.1, unmodified upstream) streams the 40x384 routed experts from CPU/NVMe, the 18
DeepSeek-V4.1-Flash-1017 sglang NVIDIA CMP 170HX 64GB ×1 (Ampere) 10.1 tok/s decode · 59.7 tok/s prefill
GLM-5.3-Flash (turboderp's EXL3 2.60 bpw pack, sc_2.60bpw) served by AlexBrandes' vLLM fork at PP2/TP1 - VLLM_PP_LAYER_PARTITION=23,22, 23 target layers on card 1 and 22 plus the D
Qwen3.8-Flash-Next (Swift 1.5 W4A16-AWQ, DotH MTP head retrained on agent traffic) on two unlocked 64 GB CMP 170HX cards (300 W VBIOS cross-flash) in vLLM pipeline parallel. PP2 (2
DeepSeek-V4.1-Flash-1017 (748B MoE) on 4x CMP 170HX (sm_80, 64 GB each, PCIe Gen2 x16), TP4, as a fidelity-first variant of the GNmac-lab recipe 1.2.0 @ 2ad5741ab1 with four change
Qwen3.8-Flash-Next (125B MoE) INT4-expert W4A16 on two CMP 170HX cards, full 262,144-token context: 241/350/198 tok/s one user (structured/code/prose), 495 tok/s editing code, 895/
Showing 12 of 33 recipes on this page — filter or paginate for the rest.