The proven 3-card pipeline-parallel recipe on CMP 170HX (sm_80, 64GB unlocked): PP=3 with manual layer partition 15,16,12 at util 0.98, FP8 KV, block-size 256, DSpark block draftin
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
13 recipes (2 beta hidden) · page 1 of 2
Ampere (1)
Community builds
Blackwell (6)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
DeepSeek-V4.1-Flash-1017 (diffbot's EXL3 3bpw pack) on 2x RTX PRO 6000 96 GB with only 128 GB of host RAM instead of the ~256 GB diffbot's own recipe pins. No fork, no rebuild: fiv
DeepSeek-V4.1-Flash-1017 vllm NVIDIA RTX PRO 6000 Blackwell 96GB ×2 167.3 tok/s decode · 7360 tok/s prefill
GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3
DeepSeek-V4.1-Flash sglang NVIDIA DGX Spark 128GB ×4 (Blackwell) 89.7 tok/s decode · 5398 tok/s prefill
Community image TP2 DSpark K3 Vision serving recipe: B12X sparse MoE, FP8 KV, three draft layers, 1M context. Saved numbers from the KK comparison.
DeepSeek-V4-Flash-Vision-Exp vllm NVIDIA RTX PRO 6000 Blackwell 96GB ×2 169.2 tok/s decode · 8993 tok/s prefill
Community image TP2 DSpark K5 serving recipe: B12X sparse MoE, FP8 KV, InstantTensor loading, 1M context. Saved numbers from the KK comparison (Max-Q +6000).
DeepSeek-V4-Flash-0731 vllm NVIDIA RTX PRO 6000 Blackwell 96GB ×2 214.9 tok/s decode · 11167 tok/s prefill
Recipe by Alex Ellis / OpenFaaS Ltd. GLM-5.3-Flash (320B-A18B, NVFP4) served tensor-parallel 4 across four DGX Spark (GB10, sm_121) nodes cabled as a closed switchless RoCE ring: t
Ampere (5)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
MTP-4 production recipe for Qwen3.8-Flash-Next-Uncensored (leoncca AWQ-g32) on 3x CMP 170HX (SM80 cards unlocked to 64 GiB): PP3 partition 17,17,14 (no GPU P2P on this pairing; the
320B MoE served on ONE 24 GB RTX 3090 with stock exllamav3 1.5.1 plus process-local monkeypatches: the 12,096 routed experts (42 MoE layers x 288, 8 picked per token per layer) tie
Validated SGLang serving for the dense Qwen3.8-27B hybrid GDN vision-language model (48 Gated-DeltaNet + 16 gated-attention layers) on RTX 3090 24 GB: one digest-pinned container w
Working PP3 recipe for Qwen3.8-Flash-Next (Qwen4Exp 180B-A6B MoE, Intel INT4 W4A16 AutoRound) on 3x CMP 170HX (SM80, unlocked 64 GiB each): the 95.4 GiB BF16 PLE n-gram table strea
Single-card RTX 3090 recipe for Qwen3.8-Flash-Next (125B MoE) in the albucino W4A16-FP8PLE build (@ ef55414, same checkpoint as the 2x3090 runtime): the GPU holds attention + dense
Showing 12 of 13 recipes on this page — filter or paginate for the rest.