{"meta":{"count":13,"page":1,"pages":2,"per_page":12,"applied":{"engine":null,"model":null,"quant":null,"storage":null,"hardware":["dgx-spark","170hx-x3","170hx-x6"],"generation":null,"task":null,"tier":null,"tags":[],"vram":null,"search":null,"sort":"updated","featured":false},"origin":"https://localairecipes.com","openapi":"/openapi.json","updated_at":"2026-10-31T16:35:31+00:00"},"recipes":[{"slug":"qwen3-8-flash-next-w4a16-autoround-dgx-spark-vllm-mtp4-iter7g","title":"Qwen3.8-Flash-Next W4A16-AutoRound DGX Spark vLLM MTP4 Iter7g","summary":"Recipe by kyaky. Qwen3.8-Flash-Next W4A16-AutoRound hybrid (INT4 experts + FP8 side layers + INT8 head; 51B-token n-gram PLE table on NVMe mmap) on one DGX Spark (GB10, 121 GB unified memory) on a verified-deterministic vLLM floor (image qwen38-flash-dgx:iter7g-20261015 on the digest-pinned vllm/vllm-openai:qwen38-flash-next base): MoE align arrival-order shim, drafter-Marlin atomicAdd fix, pinned triton config, compile-cache mount. Measured line v19a-w5 = MTP=4 window-5 block verification: 76.8 tok/s single-stream, 261.2 tok/s aggregate at 8 streams (+9.2% N1 / +4.8% N8 over MTP=3, two boots per arm, same image). Promoted serving line v20-fp8: fp8 KV cache on a 24 GB pool at native 262,144 context, 1,291,120-token capacity. Chinese +31.3% via a 25,862-id (39.5%) Han draft-vocab rebalance (prose-zhen 34.8 to 45.7 tok/s). Stochastic-correctness gate 37/37 vs bit-exact fp64 oracle; per-round raw data + scripts ship in-repo; copy-agentic-1 failure disclosed.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":61.2999999999999971578290569595992565155029296875,"prefill_tok_s":0,"updated_at":"2026-10-29 21:32:28.107205-06","commit_sha":"dc8982acabf5e6b6417f4871ad5149d61f01755d","content_hash":"e6a2cfdf4903c7ca1b80df3d0f496d3ca3f13840","votes":0,"credit":[{"name":"kyaky","url":"https://github.com/kyaky/qwen3.8-flash-next-gb10-ultrafast"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-w4a16-autoround-dgx-spark-vllm-mtp4-iter7g.json","alternate":"/r/qwen3-8-flash-next-w4a16-autoround-dgx-spark-vllm-mtp4-iter7g"}},{"slug":"qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-nextn-mtp","title":"Qwen3.8-Flash-Next NVFP4 DGX Spark GB10 SGLang NEXTN MTP","summary":"Production single-node DGX Spark (Grace Blackwell GB10, SM121, 128GB unified memory) recipe for Qwen3.8-Flash-Next on SGLang: ModelOpt NVFP4 quantization with zero-copy NVMe mmap of the 47.7 GiB ple_table.bin via --ple-offload-embedding (eliminates 51 GiB of unified-memory footprint), custom Triton FP4 prefill kernels + trtllm_mha decode kernels on SM121, NEXTN MTP (3 verification steps / 4 draft tokens) lifting output to 110-152 tok/s, GDN+QSA hybrid (131,072-token served window, 262k native capacity), 96-slot mamba recurrence buffers, sharp froggeric v22.1 Jinja chat template (-18% response length), persistent Triton/Inductor SM121 JIT caches on NVMe, 8-worker parallel NVMe pre-warm (206 shards in 17s) + 16-core Grace OMP. Author-measured vs same-box 27B dense baseline: 110.4-152.8 tok/s decode w/ MTP (baseline 12.1 base / 21.5 MTP), single-stream CoT 25.1-28.8 tok/s (2.62x speculative gain), TTFT ~347ms radix hit / ~465ms cold, 113 GiB / 121 GiB RAM used (~8 GiB headroom), 103.5 GiB","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":110.400000000000005684341886080801486968994140625,"prefill_tok_s":0,"updated_at":"2026-10-29 21:32:27.869274-06","commit_sha":"2c5220aa85b5379d7373abd0aa14bb13b1ec993f","content_hash":"bf0c02f95b99494d6720a515e2f8f438f126bc87","votes":0,"credit":[{"name":"Michael Bemler (Bemler Labs)","url":"https://github.com/bemlerlabs/qwen3.8-flash-next-dgx-spark-sglang"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-nextn-mtp.json","alternate":"/r/qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-nextn-mtp"}},{"slug":"qwen3-8-flash-next-fp4-local-1x-dgx-spark-vllm-b12x","title":"Qwen3.8-Flash-Next FP4-Local 1x DGX Spark vLLM b12x","summary":"Recipe by sojufx. Serves NVIDIA's nvidia/Qwen3.8-Flash-Next-FP4-Local (96 GiB NVFP4 quantization-aware-distilled checkpoint in a new compressed NVFP4 scale format) on one DGX Spark (GB10) via the digest-pinned eugr/spark-vllm-b12x community container (reported vLLM 0.1.dev21830+g8244d1c44.d20261015) - the model card requires community b12x containers until the format lands upstream. FP8 KV cache, exact MTP K=3 with standard rejection sampling, 262,144-token context, 4 scheduler slots, 4096-token batched prefill, disk-backed io_uring PLE, prefix cache with Mamba align, vision plus qwen3_xml tool calling, thinking off by default, localhost-only 127.0.0.1:8001 OpenAI-compatible backend with a deliberate no-replace/no-port-take guard. Measured 2026-10-15/16 on the author's Spark: 66.2 tok/s single-stream sparkDash counting decode, 166.4 tok/s at C4, 41-54 tok/s single-stream on the four-prompt wall-clock suite - different measurement bases, not interchangeable.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":66.2000000000000028421709430404007434844970703125,"prefill_tok_s":0,"updated_at":"2026-10-29 21:32:27.177524-06","commit_sha":"5fc0d101d8e9b1c6a1419f3d465e21649f4ecbad","content_hash":"be42677d0cc5ca7f71fe72210e756f1622fe3604","votes":0,"credit":[{"name":"sojufx","url":"https://github.com/sojufx/Qwen3.8-Flash-Next-FP4-Local-DGX-Spark-Recipe"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-fp4-local-1x-dgx-spark-vllm-b12x.json","alternate":"/r/qwen3-8-flash-next-fp4-local-1x-dgx-spark-vllm-b12x"}},{"slug":"nemotron-3-5-bolt-74b-a9b-1x-dgx-spark-vllm-b12x-lilicorr-262k","title":"Nemotron 3.5 Bolt 74B-A9B 1x DGX Spark vLLM b12x LiLiCorr 262K","summary":"NVIDIA Nemotron 3.5 Bolt 74B-A9B-24GB (hybrid Mamba-2/Transformer latent-MoE, 74B with 9B active, IQ2_XXS 2-bit experts, NVFP4 Mamba layers, Q8_0 rest; text+image+video in, 262,144-token window, OpenMDW-1.1) on one DGX Spark (GB10/SM121) with vLLM nightly-77154ca6 (base image vllm/vllm-openai@sha256:1ee85ca8) + b12x 1.5.1 kernels (--linear-backend b12x --moe-backend b12x), FP8 KV cache, gpu-memory-utilization 0.70, max-num-seqs 16, and the NVIDIA LiLiCorr MTP drafter at 7 speculative tokens. Pinned revisions: model c6cc2e8f, drafter 4c6a5613. Measured on the reference box: ~140 tok/s code rewrite / 63 summarise / 50 prose single-stream with the drafter (mean acceptance 4.0 tokens/pass), 31 tok/s without; 74.8/113.4/163.6/201.2 tok/s at 1/4/8/16 streams; KV cache ~6.52M tokens at the 0.70 share. On a 64 GB Spark the lane takes its own memory share 0.60: 667,874 KV tokens (2.55 windows), 110.8 tok/s greedy, 7.1 GiB left at the peak (2026-10-24).","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":63,"prefill_tok_s":0,"updated_at":"2026-10-29 21:32:26.717952-06","commit_sha":"2cdb5f11ffa1fb1af8676496fdd631976c3d536d","content_hash":"28887fb2cf3bab1914c803e792898fff92b9afe0","votes":0,"credit":[{"name":"hasso5703","url":"https://github.com/hasso5703"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/nemotron-3-5-bolt-74b-a9b-1x-dgx-spark-vllm-b12x-lilicorr-262k.json","alternate":"/r/nemotron-3-5-bolt-74b-a9b-1x-dgx-spark-vllm-b12x-lilicorr-262k"}},{"slug":"glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4","title":"GLM-5.3-Flash exlr8 EXL3 experts on DGX Spark TP1-4","summary":"GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3/4 bpw), chosen at boot via VLLM_EXLR8_WIDTHS. Reference topology TP=4 on four GB10 boxes; TP=1 serves the whole model (~87 GiB) on one 128 GB DGX Spark / ASUS GX10 via tp1.yaml plus a distinct MENTAT_GROUP. TP=2 and TP=3 also measured. 2026-10-19, clocks TP-locked at 2200 MHz: decode code 128.0 tok/s, prose 72.5 at TP=4; 53.0 / 29.5 at TP=1; prefill 4,393 tok/s at 8K, 4,190 at 32K. Trade vs the shipping NVFP4 build: decode +3-8%, NLL equal or better, KV pool ~+15%, prefill 7.7% lower at Second 1x-Spark GLM-5.3-Flash lane: not the VQ Q4_K GGUF lane (/r/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn), and not a fork of the 2x ATLAS build (/r/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m) whose NVFP4 Flash base this stack overlays.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":65.2000000000000028421709430404007434844970703125,"prefill_tok_s":3438.5,"updated_at":"2026-10-29 21:32:25.385656-06","commit_sha":"2ddd4d836d8efa003f7203ba30597c20dc21e495","content_hash":"79db03010b10026e9788b7c749df25daf5fb38be","votes":0,"credit":[{"name":"kindlingai","url":"https://github.com/kindlingai/glm-5.3-flash-exlr8-gx10"},{"name":"nvidia","url":"https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4.json","alternate":"/r/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4"}},{"slug":"ds4-flash-3x170hx-pp3-dspark-k5","title":"DeepSeek V4 Flash-0731 abliterated on 3x CMP 170HX \u2014 vLLM patched PP3 DSpark K5","summary":"The proven 3-card pipeline-parallel recipe on CMP 170HX (sm_80, 64GB unlocked): PP=3 with manual layer partition 15,16,12 at util 0.98, FP8 KV, block-size 256, DSpark block drafting at exactly 5 speculative tokens, async scheduling, CPU pinning (dedicated physical cores, placeholder in launch args), hard memory cap 64g with swap disallowed. Structured decode 181 tok/s at 99% acceptance. Upstream reference repo: allover326/deepseek-v4-cmp170hx @ 3dd2d88 (Aug 13 2026) \u2014 documents 15,15,13 3-card split, retracted 4-card requirement, INT4 524288 ctx w/ DSV4_LOGITS_ROW_CHUNK=64, +7% decode via c3046d1 rebase patches (full source build required). First-party 15,16,12 numbers on this page are ours.","tier":"curated","beta":false,"generation":"ampere","decode_tok_s":129.900000000000005684341886080801486968994140625,"prefill_tok_s":2350,"updated_at":"2026-10-29 21:32:23.860633-06","commit_sha":"24a9ac90156459e4efb49c951590edbea775df38","content_hash":"02d56ece2249f2b3e2b1ee2ffce158e841e20d26","votes":0,"credit":[{"name":"allover326","url":"https://github.com/allover326/deepseek-v4-cmp170hx"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/ds4-flash-3x170hx-pp3-dspark-k5.json","alternate":"/r/ds4-flash-3x170hx-pp3-dspark-k5"}},{"slug":"qwen3-8-27b-sglang-nvfp4-dflash2-on-1x-dgx-spark-deterministic","title":"Qwen3.8-27B SGLang NVFP4 DFlash2 on 1x DGX Spark (deterministic)","summary":"First dense Qwen3.8-27B recipe on 1x DGX Spark (GB10, 128 GB unified): SGLang v0.5.22 release image + NVFP4 target + DFlash2 speculative decoding (NVFP4 calibrated 5-layer draft, depth 16), served by a boot-persistent systemd unit from a one-command installer. Author-measured 68.3 tok/s greedy median single stream (bench.sh, thinking on; v1.39.0 A/B 2026-10-23), 81.7 tok/s at 4 streams (serving-bench), 135-148 tok/s at 8 streams and 258 at 32. Fully pinned (image digest, checkpoint revisions, draft depth) and reproducible to the decimal across boots via --disable-flashinfer-autotune (\"The boot lottery\", BENCHMARKS.md); 1M-token YaRN context by default, native 262144 as fallback; OpenAI- and Anthropic-compatible API on :30001 through the box's keepalive proxy (the engine itself answers on 127.0.0.1:30000, on the box only), with opencode wired out of the box.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":68.2999999999999971578290569595992565155029296875,"prefill_tok_s":0,"updated_at":"2026-10-28 16:49:58.824279-06","commit_sha":"307fa2334d111e38258e27fa9e690c44e96b441b","content_hash":"7d8b3e291e1b8abfa9db4ded3e90cd601b8bba78","votes":0,"credit":[{"name":"hasso5703","url":"https://github.com/hasso5703"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-27b-sglang-nvfp4-dflash2-on-1x-dgx-spark-deterministic.json","alternate":"/r/qwen3-8-27b-sglang-nvfp4-dflash2-on-1x-dgx-spark-deterministic"}},{"slug":"qwen3-8-flash-next-nvfp4-1x-dgx-spark-vllm-71-3-c1-salient-data","title":"Qwen3.8-Flash-Next NVFP4 1x DGX Spark vLLM 71.3 c1 (salient-data)","summary":"Qwen3.8-Flash-Next (~180B multimodal MoE, NVFP4) on one NVIDIA DGX Spark (GB10, 121 GiB unified memory) with a pinned vLLM nightly image + 14 provenance-tracked patches (patch --fuzz=0) + one docker run, fitting the 51.2 GB PLE n-gram table by never making it resident (rows read per forward-pass step from the checkpoint file). 69.3 tok/s single-stream median decode (runs 66.8/69.5/69.3), 41.0 prose median, 91.9 tok/s 6-stream aggregate (1 run), GSM8K 99/100 on a fixed 100-problem subset (temp 0.6 top-p 0.95 thinking on), 12/12 arithmetic + 131K needle pass + prefix-cache canary pass, prefix caching ON. Measured with tonyd2wild's bench_categories.py @ 6ad1c8f unchanged on build 2026-10-22-8e2beaa3; clean first-time run 2026-10-25 re-measured 69.8/41.5/82.6. Prod/dev: prod image digest-pinned + cosign-signed + SPDX SBOM; dev rebuilds the same serving code from published patches with an AST-equivalence proof. Outside the model card's support statement (B200/B300 listed, GB10 unlisted).","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":69.2999999999999971578290569595992565155029296875,"prefill_tok_s":0,"updated_at":"2026-10-28 15:59:39.152527-06","commit_sha":"85e2e4120ef69829bff82736c39745ac579c9a6b","content_hash":"63f86642d07e8833929005e2087044adaecc1ca4","votes":0,"credit":[{"name":"Jeremy Newhouse / salient-data (recipe, patches, measurements)","url":"https://github.com/salient-data/qwen38-next-dgx-recipe"},{"name":"tonyd2wild (bench_categories.py @ 6ad1c8f + comparator stack/numbers)","url":"https://github.com/tonyd2wild/Qwen3.8-Flash-Next-NVFP4-DGX-Spark"},{"name":"NVIDIA (Qwen3.8-Flash-Next-NVFP4 checkpoint; HF token gated)","url":"https://huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4"},{"name":"vLLM project (nightly base image 8e2beaa3)","url":"https://github.com/vllm-project/vllm"},{"name":"Sigstore/cosign (image signing + SBOM timestamping)","url":"https://docs.sigstore.dev"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-nvfp4-1x-dgx-spark-vllm-71-3-c1-salient-data.json","alternate":"/r/qwen3-8-flash-next-nvfp4-1x-dgx-spark-vllm-71-3-c1-salient-data"}},{"slug":"deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5","title":"DeepSeek-V4.1-Flash SM80 DeepInfer kernel port 6x CMP 170HX DSpark K5","summary":"DeepSeek-V4.1-Flash served on six 64 GB SM80 GPUs (GA100 die, owner-classified 170HX) with HalfVulpes' Deepinfer-SM80 patch set: a 30-file vLLM V1 overlay porting DeepSeek's DeepInfer kernel architecture (SM100 tcgen05/TMEM) to Ampere mma.sync tensor cores - unified 512-dim SM80 sparse-MLA decode/prefill kernels with bitshift UE8M0 scale decoding, a bit-exact single-CTA Triton TopK+SWA combiner (22.17x), direct-store KV-cache gather (7.64x @48K), SM80-aligned TileLang mHC routing (2.89x), and YOCO L20-L39 ragged indexer-metadata reuse (19 redundant kernel chains eliminated per chunk). TP2 x PP3 (14/14/12) + DSpark K=5 + 1M context, 4.65M-token GPU-resident KV. Author-measured: 64K cold prefill 4,507 tok/s (+24.2% vs baseline vLLM SM80), 64K TTFT 10.7 s (-18.9%), c1 DSpark decode 160.8 tok/s code / 188.8 tok/s 16K+ long-gen, c4 170.5 aggregate. Microbenchmarks bit-exact (0 diff) or within BF16 noise. Topology guide: PP6 = 2.1x slower single-stream; TP2+TP4 asymmetric hurts Marlin tiles.","tier":"community","beta":false,"generation":"ampere","decode_tok_s":160.80000000000001136868377216160297393798828125,"prefill_tok_s":4506.8000000000001818989403545856475830078125,"updated_at":"2026-10-28 15:48:17.404025-06","commit_sha":"ced28259b90a43ec103ecd450b68455a56d2808d","content_hash":"e00265eca8b84734c2c7de2b67d558b0d702d25e","votes":0,"credit":[{"name":"HalfVulpes (Deepinfer-SM80 port + measurements)","url":"https://github.com/HalfVulpes/Deepinfer-SM80"},{"name":"DeepSeek-AI (DeepInfer engine architecture + DeepSeek-V4.1-Flash checkpoint)","url":"https://github.com/deepseek-ai/DeepInfer"},{"name":"vLLM project (V1 engine + TPxPP runtime)","url":"https://github.com/vllm-project/vllm"},{"name":"TileLang + OpenAI Triton (SM80 kernel synthesis)","url":"https://github.com/tile-ai/tilelang"},{"name":"FlashInfer + IST-DASLab Marlin (sparse-MLA + MXFP4/W4A16 Ampere GEMM primitives)","url":"https://github.com/flashinfer-ai/flashinfer"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5.json","alternate":"/r/deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5"}},{"slug":"glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn","title":"GLM-5.3-Flash VQ Q4_K 1x DGX Spark 1M ctx (JLFN)","summary":"Recipe by JLFN (dgx-recipe, verified on-box 2026-10-17). Baekpica/GLM-5.3-Flash-VQ-GGUF (72 GiB GVQ1 release directory: routed experts vector-quantized with codebook, 316 nonexpert matrices Q4_K, rest source precision; 56 GGUF parts, embedded MTP predictor) served by the ds4-dfm-rs Rust host (ds4-server, CUDA) on one DGX Spark at the model's native 1048576-token context with two banks. Measured at temperature 0: 24.2-24.3 tok/s one stream, 12.1/12.0 each with two streams (aggregate flat - decode is bytes-per-token against the weight stream). Disclosed quality trade: prose PPL 4.8555 vs 8.0797, code-completion 10.5 vs 18.4%, Korean meeting-summary degeneration under greedy.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":24.300000000000000710542735760100185871124267578125,"prefill_tok_s":0,"updated_at":"2026-10-20 21:52:36.476254-06","commit_sha":"f5a53e8b06f37327380e8affb1c9ea298e270456","content_hash":"2d63af307f133a15a2baacaee8d45c16f33ff129","votes":0,"credit":[{"name":"JLFN","url":"https://github.com/JLFN/dgx-recipe"},{"name":"Baekpica","url":"https://huggingface.co/Baekpica"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn.json","alternate":"/r/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn"}},{"slug":"qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle","title":"Qwen3.8-Flash-Next W4A16 3x CMP 170HX PP3 NVMe PLE fp8 KV 1M (clee)","summary":"Working PP3 recipe for Qwen3.8-Flash-Next (Qwen4Exp 180B-A6B MoE, Intel INT4 W4A16 AutoRound) on 3x CMP 170HX (SM80, unlocked 64 GiB each): the 95.4 GiB BF16 PLE n-gram table streams live from NVMe (shard-direct preadv into a fixed GPU staging buffer, never fully loaded), fp8 e4m3 KV software-decoded on SM80, mamba SSM cache bf16, and 1,048,576-token context via YaRN x4 over the 262144-native checkpoint. KV pool 5.97M tokens (~5.7 concurrent 1M sessions). W4A16 experts run GPTQ-Marlin bit-exact; no GPU P2P on this card pairing, so PP stages through host RAM with NCCL_P2P_DISABLE=1 + --disable-custom-all-reduce. Measured cold-cache temp=0: prefill 2389 tok/s @4k rising to 4690 @1M (QSA sparse crossover past ~64k), decode 73.2 -> 59.8 tok/s @1M, TTFT 223 s @1M; exact needle recall at all depths through 1M. Fork clee/vllm-sm80 on a PINNED nightly; newer nightlies not validated.","tier":"community","beta":false,"generation":"ampere","decode_tok_s":73.2000000000000028421709430404007434844970703125,"prefill_tok_s":3563,"updated_at":"2026-10-20 15:34:25.555323-06","commit_sha":"fb3da33b3299da70eb24ce68a2c772de0dc1a356","content_hash":"0dea5f8b482a14a47e6cef76e85ad322d56f9821","votes":1,"credit":[{"name":"clee (recipe, fork patches, bench + needle harness)","url":"https://github.com/clee/qwen3.8-flash-next-3x170hx"},{"name":"clee (vllm-sm80 fork: units-fix, fp8-kv-sm80, ple-pp3, ple-nvme-staged)","url":"https://github.com/clee/vllm-sm80"},{"name":"Kai / 2Wild (Qwen38-Flash-Next-4x3090 \u2014 staged-PLE architecture inspiration)","url":"https://github.com/tonyd2wild/Qwen38-Flash-Next-4x3090"},{"name":"Intel (INT4 W4A16 AutoRound quant)","url":"https://huggingface.co/Intel/Qwen3.8-Flash-Next-W4A16-AutoRound"},{"name":"froggeric (bugfixed Qwen3.8 chat templates)","url":"https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle.json","alternate":"/r/qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle"}},{"slug":"qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2","title":"Qwen3.8-Flash-Next Uncensored NVFP4 BF16 PLE DGX Spark vLLM MTP2","summary":"DGX Spark recipe running NVFP4 Qwen3.8-Flash-Next with its FULL-PRECISION BF16 PLE memory-mapped from NVMe on vLLM (NOT the FP8-table variants the other Spark recipes use): the complete 95.37 GiB BF16 PLE stays in the checkpoint on NVMe and is mmap'd, gathering only the rows each request needs, while the NVFP4 compute trunk stays in unified memory - PIECEWISE CUDA graphs + BF16 KV cache. Runtime-qualified ONLY for orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 @ 3a3b63161c0745390e5270179af42e46efc70799 (gated HF repo) plus the recipe's deterministic overlay of exactly 31 native BF16 MTP tensors sourced from pinned RadixArk/Qwen3.8-Flash-Next-NVFP4 @ 7b719225242aacd3dbd3f9407468c2ee9a9d2594 (Orcarouter ships zero MTP tensors despite advertising an MTP layer; 4.86 GiB compact BF16 MTP file; load guard verifies 31/31 tensor identities, hash+dtype+name+arch+load-count guarded - checkpoint-specific, not a generic plug-in). Engine: official vllm/vllm-openai:qwen38-flash-next image @ sha256:f","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":44.2000000000000028421709430404007434844970703125,"prefill_tok_s":2055.1999999999998181010596454143524169921875,"updated_at":"2026-10-20 15:34:13.655428-06","commit_sha":"fbfeac5f3b0f82db6cddd8164911dcb3173edf1d","content_hash":"89ee2463a76211f23776cd44e1d72def5dd56dc5","votes":0,"credit":[{"name":"YSLAB-ai (BF16 PLE support, DGX Spark fixes, guarded MTP overlay, runtime qualification)","url":"https://github.com/YSLAB-ai/Qwen3.8-Flash-Next-NVFP4-BF16PLE-DGX-Spark"},{"name":"blazux (originated the FP8 PLE mmap work)","url":"https://github.com/blazux"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2.json","alternate":"/r/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2"}}]}