Recipe by kyaky. Qwen3.8-Flash-Next W4A16-AutoRound hybrid (INT4 experts + FP8 side layers + INT8 head; 51B-token n-gram PLE table on NVMe mmap) on one DGX Spark (GB10, 121 GB unif
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
14 recipes · page 1 of 2
Community builds
Blackwell (10)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Production single-node DGX Spark (Grace Blackwell GB10, SM121, 128GB unified memory) recipe for Qwen3.8-Flash-Next on SGLang: ModelOpt NVFP4 quantization with zero-copy NVMe mmap o
Recipe by sojufx. Serves NVIDIA's nvidia/Qwen3.8-Flash-Next-FP4-Local (96 GiB NVFP4 quantization-aware-distilled checkpoint in a new compressed NVFP4 scale format) on one DGX Spark
NVIDIA Nemotron 3.5 Bolt 74B-A9B-24GB (hybrid Mamba-2/Transformer latent-MoE, 74B with 9B active, IQ2_XXS 2-bit experts, NVFP4 Mamba layers, Q8_0 rest; text+image+video in, 262,144
GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3
First dense Qwen3.8-27B recipe on 1x DGX Spark (GB10, 128 GB unified): SGLang v0.5.22 release image + NVFP4 target + DFlash2 speculative decoding (NVFP4 calibrated 5-layer draft, d
Qwen3.8-Flash-Next (~180B multimodal MoE, NVFP4) on one NVIDIA DGX Spark (GB10, 121 GiB unified memory) with a pinned vLLM nightly image + 14 provenance-tracked patches (patch --fu
Pipeline-parallel TP1xPP2 + DSpark serving of DeepSeek V4.1 Flash (1017 official FP8 weights) across two DGX Station GB300 nodes over 400G CX8 RoCE. PP2 splits the 40 layers 20/20
Recipe by JLFN (dgx-recipe, verified on-box 2026-10-17). Baekpica/GLM-5.3-Flash-VQ-GGUF (72 GiB GVQ1 release directory: routed experts vector-quantized with codebook, 316 nonexpert
DGX Spark recipe running NVFP4 Qwen3.8-Flash-Next with its FULL-PRECISION BF16 PLE memory-mapped from NVMe on vLLM (NOT the FP8-table variants the other Spark recipes use): the com
Qwen3.8-Flash-Next vllm NVIDIA DGX Spark 128GB ×1 (Blackwell) 44.2 tok/s decode · 2055.2 tok/s prefill
Ampere (2)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
320B MoE served on ONE 24 GB RTX 3090 with stock exllamav3 1.5.1 plus process-local monkeypatches: the 12,096 routed experts (42 MoE layers x 288, 8 picked per token per layer) tie
Validated SGLang serving for the dense Qwen3.8-27B hybrid GDN vision-language model (48 Gated-DeltaNet + 16 gated-attention layers) on RTX 3090 24 GB: one digest-pinned container w
Showing 12 of 14 recipes on this page — filter or paginate for the rest.