{"slug":"qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2","title":"Qwen3.8-Flash-Next Uncensored NVFP4 BF16 PLE DGX Spark vLLM MTP2","summary":"DGX Spark recipe running NVFP4 Qwen3.8-Flash-Next with its FULL-PRECISION BF16 PLE memory-mapped from NVMe on vLLM (NOT the FP8-table variants the other Spark recipes use): the complete 95.37 GiB BF16 PLE stays in the checkpoint on NVMe and is mmap'd, gathering only the rows each request needs, while the NVFP4 compute trunk stays in unified memory - PIECEWISE CUDA graphs + BF16 KV cache. Runtime-qualified ONLY for orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 @ 3a3b63161c0745390e5270179af42e46efc70799 (gated HF repo) plus the recipe's deterministic overlay of exactly 31 native BF16 MTP tensors sourced from pinned RadixArk/Qwen3.8-Flash-Next-NVFP4 @ 7b719225242aacd3dbd3f9407468c2ee9a9d2594 (Orcarouter ships zero MTP tensors despite advertising an MTP layer; 4.86 GiB compact BF16 MTP file; load guard verifies 31/31 tensor identities, hash+dtype+name+arch+load-count guarded - checkpoint-specific, not a generic plug-in). Engine: official vllm/vllm-openai:qwen38-flash-next image @ sha256:f","license":"CC-BY-4.0","tier":"community","beta":false,"revision":{"no":2,"commit_sha":"fbfeac5f3b0f82db6cddd8164911dcb3173edf1d","content_hash":"89ee2463a76211f23776cd44e1d72def5dd56dc5","created_at":"2026-10-20 15:34:13.655428-06","notes":"Ingested from GitHub repo YSLAB-ai/Qwen3.8-Flash-Next-NVFP4-BF16PLE-DGX-Spark @ 9492b78c6bc04638587b42f2ffeb0680b0fcb3ef (wave 3, kanban t_21a3a083). Same dgx-spark hardware family as bemlerlabs/Felliks (ONE profile created for all three, per card dedupe instruction) but DIFFERENT engine lane: this is the vLLM one (official qwen38-flash-next image) vs their SGLang. Distinct checkpoint row: orcarouter Uncensored-NVFP4 (gated) vs the site's existing RadixArk NVFP4 row. Recipe code Apache-2.0; weights under their own licenses/access terms (NOTICE)."},"engine_source":{"type":"docker_image","git_url":"https://github.com/YSLAB-ai/Qwen3.8-Flash-Next-NVFP4-BF16PLE-DGX-Spark","base_engine":"vllm","pip_package":null,"base_version":"0.1.dev20073+g8e685d198 (official qwen38-flash-next image)","image_digest":"vllm/vllm-openai:qwen38-flash-next@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8","install_note":"docker build -t qwen38-flash-dgx . per repo Dockerfile @ 9492b78c: FROM vllm/vllm-openai:qwen38-flash-next @ sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 (multi-arch, arm64 for the Spark's Grace CPU) + COPY vllm_ple_mmap.py + 5 patch scripts applied to the in-image vLLM sources (config, quantized lm_head + MTP, parallel lm_head linear attrs, MTP compressed-ignore, MTP load guard) + ple_layer.py hook appended (python3 -c ast.parse guard over all 5 patched files in the build); image label ai.yslab.vllm.version=0.1.dev20073+g8e685d198. Serving driven by the recipe CLI (scripts/qwen38-dgx-spark -> python -m recipe.cli): login (stores token in recipe cache), prepare orca-uncensored-bf16-mtp (downloads pinned checkpoint + only the 3 pinned source shards holding the 31 MTP tensors, writes audited local overlay; pre-requests config.json so gated-access failures happen early), serve orca-uncensored-bf16-mtp --mtp 2.","pinned_commit":"9492b78c6bc04638587b42f2ffeb0680b0fcb3ef"},"credit":[{"name":"YSLAB-ai (BF16 PLE support, DGX Spark fixes, guarded MTP overlay, runtime qualification)","url":"https://github.com/YSLAB-ai/Qwen3.8-Flash-Next-NVFP4-BF16PLE-DGX-Spark"},{"name":"blazux (originated the FP8 PLE mmap work)","url":"https://github.com/blazux"}],"author":{"name":"localai-recipes","isSite":false},"uploaded_by":"localai-recipes","config":{"launch":["vllm","serve","/recipe/cache/orca-uncensored-bf16-mtp","--served-model-name","qwen3.8-flash-next-orca-uncensored-bf16-mtp","--host","0.0.0.0","--port","8000","--load-format","safetensors","--max-model-len","262144","--max-num-seqs","8","--gpu-memory-utilization","0.80","--no-enable-prefix-caching","--enable-chunked-prefill","--max-num-batched-tokens","8192","-cc.cudagraph_mode=PIECEWISE","-cc.splitting_ops=[\"vllm::unified_attention_with_output\",\"vllm::unified_mla_attention_with_output\",\"vllm::mamba_mixer2\",\"vllm::mamba_mixer\",\"vllm::short_conv\",\"vllm::qwen3_8_flash_next_ple_short_conv\",\"vllm::qwen3_8_flash_next_qsa_with_output\",\"vllm::linear_attention\",\"vllm::qwen_gdn_attention_core\",\"vllm::qwen_gdn_attention_core_fused_norm_packed\",\"vllm::sparse_attn_indexer\",\"vllm::ple_mmap_lookup\"]","--no-enable-flashinfer-autotune","--kv-cache-dtype","bfloat16","--enable-auto-tool-choice","--tool-call-parser","qwen3_coder","--reasoning-parser","qwen3","--speculative-config","{\"method\":\"mtp\",\"num_speculative_tokens\":2}"],"runtime":{"env":"HF_HOME=<recipe cache> | HF_HUB_OFFLINE=1 | VLLM_PLE_MMAP=1 | VLLM_PLE_MMAP_WORKERS=32 | VLLM_PLE_MMAP_PREWARM=0 | VLLM_USE_FLASHINFER_SAMPLER=1","notes":"Serve profile = src/recipe/runtime.py build_docker_command() qualified profile @ 9492b78c verbatim (docker run -d --name qwen38-flash-orca-uncensored-bf16-mtp --restart no --label ai.yslab.recipe=qwen38-flash-next-bf16-ple --gpus all --ipc=host --shm-size 16g -p 127.0.0.1:18300:8000; loopback-only by default, options.port 18300 default, options.bind 127.0.0.1). Qualified = native 262,144 ctx, 8 max sequences, 0.80 gpu-mem util, PIECEWISE capture w/ explicit splitting_ops list, BF16 KV, no PLE prewarm, MTP=2 (RuntimeOptions defaults; --unsafe-override escapes the guard). The runtime unloads an existing recipe-labelled container before a guarded replacement. Deep MTP (depth>=5) would add --block-size 48 (QSA ring 12-token capacity x 48-token kernel alignment) - NOT used in the qualified profile. Thinking sampler: temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0 / presence 0.0 / repetition 1.0, medium reasoning effort (request-side, not baked into weights). Requires >=100 GiB free disk gate + HF account that accepted the Orcarouter gated-access conditions (browser step - a read token cannot grant access until accepted). Inferact @ 103a7608316173ca6edd49929544244de7ffda70 and RadixArk hybrid-BF16 tooling-compatible but runtime UNVALIDATED (COMPATIBILITY.md matrix)."},"max_context_len":262144},"models":[{"slug":"qwen3-8-flash-next-uncensored-nvfp4-bf16ple","name":"Qwen3.8-Flash-Next-Uncensored-NVFP4-BF16PLE","hf_repo":"orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4","quant":"nvfp4"}],"hardware":[{"slug":"dgx-spark","vendor":"nvidia","card":"DGX Spark (GB10, unified memory)","card_family":"dgx-spark","vram_gb":128,"device_count":1}],"generation":"blackwell","tags":["bf16-ple","dgx-spark","gb10","mtp-overlay","nvme-mmap","orcarrouter","piecewise-cudagraph","uncensored","vllm","yslab-ai"],"tasks":["chat","code"],"aa":{"aa_index":40,"aa_provenance":"measured","aa_source_url":"https://artificialanalysis.ai/models/qwen3-8-flash-next","aa_index_version":null,"aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-06 18:30:00-06","aa_family_slug":"qwen3-8-flash-next","aa_resolved_slug":"qwen3-8-flash-next"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Author-measured (docs/BENCHMARKS.md + results/orcarouter/*.json @ 9492b78c, one DGX Spark, Orcarouter checkpoint + BF16 MTP overlay ONLY): single-stream MTP depth sweep (32,768-token profile, concurrency 1, one max sequence, 0.80 util, 128-token warm-up, 3 fixed 256-token streamed samples; primary rate = ALL completion tokens incl. hidden reasoning / wall time) - MTP0 27.26, MTP1 33.17 (65.9%), MTP2 44.23 (73.7% = 532/722), MTP3 40.95 (56.7%), MTP4 41.84 (53.2%), MTP5 35.45 (29.1%), MTP6 35.71 (\", \"tok_s\": 44.22999999999999687361196265555918216705322265625, \"points\": [{\"notes\": \"mtp-bf16-sweep.json: MTP=2 qualified 44.23 tok/s end-to-end (acceptance 73.7%); visible-phase 46.15 tok/s; TTFC 1.324s\", \"ttft_ms\": 1324, \"concurrency\": 1, \"context_len\": 32768, \"decode_tok_s\": 44.22999999999999687361196265555918216705322265625}, {\"notes\": \"mtp-bf16-sweep.json: MTP=0 baseline 27.26 tok/s (MTP2 = +62.2%)\", \"concurrency\": 1, \"context_len\": 32768, \"decode_tok_s\": 27.260000000000001563194018672220408916473388671875}, {\"notes\": \"native-240k.json: exact retrieval pass at 240,051 prompt tokens; 17.37 GiB BF16 KV = 627,960 tok capacity\", \"ttft_ms\": 116800, \"concurrency\": 1, \"context_len\": 240051, \"prefill_tok_s\": 2055.170000000000072759576141834259033203125}], \"context_len\": 32768}"}],"perf_points":[{"id":1060,"context_len":32768,"prefill_tok_s":null,"decode_tok_s":44.22999999999999687361196265555918216705322265625,"ttft_ms":1324,"concurrency":1,"claimed_by":"submitter","notes":"mtp-bf16-sweep.json: MTP=2 qualified 44.23 tok/s end-to-end (acceptance 73.7%); visible-phase 46.15 tok/s; TTFC 1.324s"},{"id":1061,"context_len":240051,"prefill_tok_s":2055.170000000000072759576141834259033203125,"decode_tok_s":null,"ttft_ms":116800,"concurrency":1,"claimed_by":"submitter","notes":"native-240k.json: exact retrieval pass at 240,051 prompt tokens; 17.37 GiB BF16 KV = 627,960 tok capacity"}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2.json"},"requirements":{"requires":["Docker (or a compatible OCI container runtime)","NVIDIA Container Toolkit (nvidia-container-toolkit) - exposes host GPUs to the container"],"driver_floor":{"cuda":null,"linux_min":null,"text":"unknown - check the image tag","source":"NVIDIA CUDA Toolkit Release Notes - driver compatibility tables (https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/, fetched 2026-10-03)"},"placeholders":["Hardware-bound values in this recipe (device counts, bind-mount paths, utilization, ports) describe the author's machine, not yours - map them to your own rig before launching. Keep the engine flags, model, quant, and parallelism strategy; adjust paths and ports to the hardware you actually have.","GPU identity: list your GPUs with `nvidia-smi -L` and substitute your own GPU UUIDs/indices wherever the recipe pins the author's.","Model directory: bind-mount paths are author-side - point them at your own mount where the model files live.","Host port: pick a free port on your machine and publish the container there; keep the in-container port unchanged."]},"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4","pin":"revision 3a3b63161c0745390e5270179af42e46efc70799 (GATED repo - accept access conditions in browser first; direct BF16 PLE, ~95.37 GiB table; zero MTP tensors despite config advertising an MTP layer)"},{"kind":"drafter","pointer_type":"hf_model","pointer":"https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4","pin":"BF16 MTP overlay SOURCE: exactly 31 native BF16 MTP tensors from pinned revision 7b719225242aacd3dbd3f9407468c2ee9a9d2594 -> 4.86 GiB compact overlay built+audited by the recipe (sha256-per-shard, hash+dtype+name+arch+load-count guards, 31/31 load-guard ve"},{"kind":"engine_image","pointer_type":"docker_digest","pointer":"https://hub.docker.com/r/vllm/vllm-openai","pin":"vllm/vllm-openai:qwen38-flash-next @ sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 + repo patch set (vllm_ple_mmap hook + 5 patch scripts); local tag qwen38-flash-dgx"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/YSLAB-ai/Qwen3.8-Flash-Next-NVFP4-BF16PLE-DGX-Spark","pin":"9492b78c6bc04638587b42f2ffeb0680b0fcb3ef - src/recipe (cli/runtime/manifest/download/audit), Dockerfile + 5 patch scripts + vllm_ple_mmap.py, compatibility.json, results/orcarouter/*.json (mtp sweep, native-240k, concurrency, functional, readiness, summary"}],"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2.json","alternate":"/r/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2","describedby":"/openapi.json"},"updated_at":"2026-10-20 15:34:13.655428-06","created_at":"2026-10-07 11:30:12.151417-06","speeds":{"decode_tok_s":44.2000000000000028421709430404007434844970703125,"prefill_tok_s":2055.1999999999998181010596454143524169921875,"leaderboard":"/api/v1/leaderboard.json"}}