Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
13 recipes (3 beta hidden) · page 1 of 2
Single-card Blackwell recipe for Qwen3.8-Flash-Next (hybrid GDN/QSA MoE, 48 layers, 512 experts) on one 96 GB RTX PRO 6000: the Community image community image R34 (localinferencel
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
DeepSeek-V4.1-Flash-1017 (diffbot's EXL3 3bpw pack) on 2x RTX PRO 6000 96 GB with only 128 GB of host RAM instead of the ~256 GB diffbot's own recipe pins. No fork, no rebuild: fiv
Recipe by AppNetDeveloper. ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ on 1x RTX PRO 6000 Blackwell (SM120) with stock SGLang 0.5.22 plus engine overlays, no fork: FR-Spec pruned draft lm_
Recipe by bill-h-lin. Qwen3.8-27B BF16 (BF16 KV, temperature 0) on one RTX PRO 6000 Blackwell Max-Q (SM120) with SGLang made bit-exact under speculative decoding: verify GDN recurr
Community image TP2 DSpark K3 Vision serving recipe: B12X sparse MoE, FP8 KV, three draft layers, 1M context. Saved numbers from the KK comparison.
Community image TP2 DSpark K5 serving recipe: B12X sparse MoE, FP8 KV, InstantTensor loading, 1M context. Saved numbers from the KK comparison (Max-Q +6000).
GLM-5.3-Flash (320B, 42x288-expert MoE) served from turboderp's EXL3 3.05bpw checkpoint (116.7 GiB) on ONE 96 GB RTX PRO 6000 Blackwell: a GPU expert cache keeps the 40 most-routed
Experimental SGLang fork for Qwen3.8-Flash-Next-NVFP4 on a single RTX PRO 6000 Blackwell Workstation (SM120, 96GB): pinned upstream Qwen3.8 (PR #36497 @ 73a255206f), NVMe-backed PL
Single-card RTX PRO 6000 Blackwell (SM120, 96GB) recipe for Qwen3.8-Flash-Next on SGLang: the 51B N-gram PLE table is read straight off NVMe with io_uring as tokens need it (measur
Local Inference Lab production two-card serving config: TP2/DCP2, 8 concurrent seqs, MTP3 on B12X, lossless-CSF NVFP4 MoE (MXFP8-CSF-QAD weights, 188.87 GB), online MXFP8 attention
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Qwen3.8-Flash-Next (125B MoE) 3 bpw AutoRound experts on ONE CMP 170HX card with the full 262,144-token context: 197.8/296.4/135.5 tok/s one user (structured/code/prose), 371 tok/s
Qwen3.8-27B at 65-67 tok/s decode (74 structured) with MTP k=3 speculative decoding AND prefix caching simultaneously on a single NVIDIA CMP 170HX 64GB (Hynix HBM2e, GA100/sm80), 2
Showing 12 of 13 recipes on this page — filter or paginate for the rest.