Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
17 recipes (1 beta hidden) · page 1 of 2
Single-card Blackwell recipe for Qwen3.8-Flash-Next (hybrid GDN/QSA MoE, 48 layers, 512 experts) on one 96 GB RTX PRO 6000: the Community image community image R34 (localinferencel
The proven 3-card pipeline-parallel recipe on CMP 170HX (sm_80, 64GB unlocked): PP=3 with manual layer partition 15,16,12 at util 0.98, FP8 KV, block-size 256, DSpark block draftin
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3
Recipe by AppNetDeveloper. ThinkingCap-Qwen3.8-27B-NVFP4A4-AWQ on 1x RTX PRO 6000 Blackwell (SM120) with stock SGLang 0.5.22 plus engine overlays, no fork: FR-Spec pruned draft lm_
Recipe by bill-h-lin. Qwen3.8-27B BF16 (BF16 KV, temperature 0) on one RTX PRO 6000 Blackwell Max-Q (SM120) with SGLang made bit-exact under speculative decoding: verify GDN recurr
Recipe by Alex Ellis / OpenFaaS Ltd. GLM-5.3-Flash (320B-A18B, NVFP4) served tensor-parallel 4 across four DGX Spark (GB10, sm_121) nodes cabled as a closed switchless RoCE ring: t
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Qwen3.8-Flash-Next (125B MoE) 3 bpw AutoRound experts on ONE CMP 170HX card with the full 262,144-token context: 197.8/296.4/135.5 tok/s one user (structured/code/prose), 371 tok/s
320B MoE served on ONE 24 GB RTX 3090 with stock exllamav3 1.5.1 plus process-local monkeypatches: the 12,096 routed experts (42 MoE layers x 288, 8 picked per token per layer) tie
Validated SGLang serving for the dense Qwen3.8-27B hybrid GDN vision-language model (48 Gated-DeltaNet + 16 gated-attention layers) on RTX 3090 24 GB: one digest-pinned container w
Working PP3 recipe for Qwen3.8-Flash-Next (Qwen4Exp 180B-A6B MoE, Intel INT4 W4A16 AutoRound) on 3x CMP 170HX (SM80, unlocked 64 GiB each): the 95.4 GiB BF16 PLE n-gram table strea
Qwen3.8-27B at 65-67 tok/s decode (74 structured) with MTP k=3 speculative decoding AND prefix caching simultaneously on a single NVIDIA CMP 170HX 64GB (Hynix HBM2e, GA100/sm80), 2
Showing 12 of 17 recipes on this page — filter or paginate for the rest.