Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
15 recipes (3 beta hidden) · page 1 of 2
Single-card Blackwell recipe for Qwen3.8-Flash-Next (hybrid GDN/QSA MoE, 48 layers, 512 experts) on one 96 GB RTX PRO 6000: the Community image community image R34 (localinferencel
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Recipe by AppNetDeveloper. ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ on 1x RTX PRO 6000 Blackwell (SM120) with stock SGLang 0.5.22 plus engine overlays, no fork: FR-Spec pruned draft lm_
Recipe by bill-h-lin. Qwen3.8-27B BF16 (BF16 KV, temperature 0) on one RTX PRO 6000 Blackwell Max-Q (SM120) with SGLang made bit-exact under speculative decoding: verify GDN recurr
Community image TP4 serving recipe: MTP3 draft, B12X MoE, FP8 KV, 1M context. Saved five-window medians (Max-Q +6000), the reference the Karmic Kraken report compares against.
Community image TP4 serving recipe: DFlash2 K7 offline-MXFP8 draft, B12X MoE, FP8 KV, 1M context. Saved five-window medians (Max-Q +6000).
Community image TP4 serving recipe: adaptive DSpark K7, RAM Engram ngram tables, 131K context cap. Saved numbers from the KK comparison.
GLM-5.3-Flash (320B, 42x288-expert MoE) served from turboderp's EXL3 3.05bpw checkpoint (116.7 GiB) on ONE 96 GB RTX PRO 6000 Blackwell: a GPU expert cache keeps the 40 most-routed
Experimental SGLang fork for Qwen3.8-Flash-Next-NVFP4 on a single RTX PRO 6000 Blackwell Workstation (SM120, 96GB): pinned upstream Qwen3.8 (PR #36497 @ 73a255206f), NVMe-backed PL
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Qwen3.8-Flash-Next (125B MoE) 3 bpw AutoRound experts on ONE CMP 170HX card with the full 262,144-token context: 197.8/296.4/135.5 tok/s one user (structured/code/prose), 371 tok/s
320B MoE served on ONE 24 GB RTX 3090 with stock exllamav3 1.5.1 plus process-local monkeypatches: the 12,096 routed experts (42 MoE layers x 288, 8 picked per token per layer) tie
Validated SGLang serving for the dense Qwen3.8-27B hybrid GDN vision-language model (48 Gated-DeltaNet + 16 gated-attention layers) on RTX 3090 24 GB: one digest-pinned container w
Qwen3.8-27B at 65-67 tok/s decode (74 structured) with MTP k=3 speculative decoding AND prefix caching simultaneously on a single NVIDIA CMP 170HX 64GB (Hynix HBM2e, GA100/sm80), 2
Showing 12 of 15 recipes on this page — filter or paginate for the rest.