Recipe by sojufx. Serves NVIDIA's nvidia/Qwen3.8-Flash-Next-FP4-Local (96 GiB NVFP4 quantization-aware-distilled checkpoint in a new compressed NVFP4 scale format) on one DGX Spark
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
6 recipes (1 beta hidden) · page 1 of 1
Community builds
Blackwell (3)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Community image TP4 serving recipe: MTP3 draft, B12X MoE, FP8 KV, 1M context. Saved five-window medians (Max-Q +6000), the reference the Karmic Kraken report compares against.
Local Inference Lab production two-card serving config: TP2/DCP2, 8 concurrent seqs, MTP3 on B12X, lossless-CSF NVFP4 MoE (MXFP8-CSF-QAD weights, 188.87 GB), online MXFP8 attention
Ampere (3)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
First EXL3 row for Qwen3.8-Flash-Next and first Morrowmake QFN lane: turboderp/Qwen3.8-Flash-Next-exl3 (EXL3 5.05 bpw) on two CMP 170HX cards (TP2, whole routed experts per card, P
2x CMP 170HX (GA100 sm_80, 64 GB/card, cmpunlocker unlock) recipe for Qwen3.8-Flash-Next from bozocentral's writeup (2026-10-07; page-reference: artifact IS the URL). Minachist INT
Qwen3.8-27B at 65-67 tok/s decode (74 structured) with MTP k=3 speculative decoding AND prefix caching simultaneously on a single NVIDIA CMP 170HX 64GB (Hynix HBM2e, GA100/sm80), 2