Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
9 recipes (2 beta hidden) · page 1 of 1
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Community image TP2 DSpark K3 Vision serving recipe: B12X sparse MoE, FP8 KV, three draft layers, 1M context. Saved numbers from the KK comparison.
Community image TP2 DSpark K5 serving recipe: B12X sparse MoE, FP8 KV, InstantTensor loading, 1M context. Saved numbers from the KK comparison (Max-Q +6000).
Local Inference Lab production two-card serving config: TP2/DCP2, 8 concurrent seqs, MTP3 on B12X, lossless-CSF NVFP4 MoE (MXFP8-CSF-QAD weights, 188.87 GB), online MXFP8 attention
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Qwen3.8-Flash-Next (125B MoE) 3 bpw AutoRound experts on ONE CMP 170HX card with the full 262,144-token context: 197.8/296.4/135.5 tok/s one user (structured/code/prose), 371 tok/s
320B MoE served on ONE 24 GB RTX 3090 with stock exllamav3 1.5.1 plus process-local monkeypatches: the 12,096 routed experts (42 MoE layers x 288, 8 picked per token per layer) tie
Validated SGLang serving for the dense Qwen3.8-27B hybrid GDN vision-language model (48 Gated-DeltaNet + 16 gated-attention layers) on RTX 3090 24 GB: one digest-pinned container w
Qwen3.8-27B at 65-67 tok/s decode (74 structured) with MTP k=3 speculative decoding AND prefix caching simultaneously on a single NVIDIA CMP 170HX 64GB (Hynix HBM2e, GA100/sm80), 2
Single-card RTX 3090 recipe for Qwen3.8-Flash-Next (125B MoE) in the albucino W4A16-FP8PLE build (@ ef55414, same checkpoint as the 2x3090 runtime): the GPU holds attention + dense
First single-CMP-170HX recipe for Qwen3.8-Flash-Next: the AutoRound 3bpw+MTP checkpoint on ONE 64 GiB SM80 GPU with the 95.37 GiB BF16 PLE table left ON the SSD — native Linux AIO