Qwen3.8-Flash-Next Uncensored NVFP4 BF16 PLE DGX Spark vLLM MTP2

community Community submission — verify on your own hardware.

recipe by YSLAB-ai (BF16 PLE support, DGX Spark fixes, guarded MTP overlay, runtime qualification)↗, blazux (originated the FP8 PLE mmap work)↗ · listed by localai-recipes

revision r2 · commit fbfeac5f3b · updated 10 d ago · license CC-BY-4.0

chat code uncensored nvme-mmap dgx-spark gb10 yslab-ai vllm bf16-ple mtp-overlay piecewise-cudagraph orcarrouter

DGX Spark recipe running NVFP4 Qwen3.8-Flash-Next with its FULL-PRECISION BF16 PLE memory-mapped from NVMe on vLLM (NOT the FP8-table variants the other Spark recipes use): the complete 95.37 GiB BF16 PLE stays in the checkpoint on NVMe and is mmap'd, gathering only the rows each request needs, while the NVFP4 compute trunk stays in unified memory - PIECEWISE CUDA graphs + BF16 KV cache. Runtime-qualified ONLY for orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 @ 3a3b63161c0745390e5270179af42e46efc70799 (gated HF repo) plus the recipe's deterministic overlay of exactly 31 native BF16 MTP tensors sourced from pinned RadixArk/Qwen3.8-Flash-Next-NVFP4 @ 7b719225242aacd3dbd3f9407468c2ee9a9d2594 (Orcarouter ships zero MTP tensors despite advertising an MTP layer; 4.86 GiB compact BF16 MTP file; load guard verifies 31/31 tensor identities, hash+dtype+name+arch+load-count guarded - checkpoint-specific, not a generic plug-in). Engine: official vllm/vllm-openai:qwen38-flash-next image @ sha256:f

Launch command

vllm serve /recipe/cache/orca-uncensored-bf16-mtp --served-model-name qwen3.8-flash-next-orca-uncensored-bf16-mtp --host 0.0.0.0 --port 8000 --load-format safetensors --max-model-len 262144 --max-num-seqs 8 --gpu-memory-utilization 0.80 --no-enable-prefix-caching --enable-chunked-prefill --max-num-batched-tokens 8192 -cc.cudagraph_mode=PIECEWISE '-cc.splitting_ops=["vllm::unified_attention_with_output","vllm::unified_mla_attention_with_output","vllm::mamba_mixer2","vllm::mamba_mixer","vllm::short_conv","vllm::qwen3_8_flash_next_ple_short_conv","vllm::qwen3_8_flash_next_qsa_with_output","vllm::linear_attention","vllm::qwen_gdn_attention_core","vllm::qwen_gdn_attention_core_fused_norm_packed","vllm::sparse_attn_indexer","vllm::ple_mmap_lookup"]' --no-enable-flashinfer-autotune --kv-cache-dtype bfloat16 --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

runtime

env: 'HF_HOME=<recipe cache> | HF_HUB_OFFLINE=1 | VLLM_PLE_MMAP=1 | VLLM_PLE_MMAP_WORKERS=32 | VLLM_PLE_MMAP_PREWARM=0 | VLLM_USE_FLASHINFER_SAMPLER=1'
notes: 'Serve profile = src/recipe/runtime.py build_docker_command() qualified profile @ 9492b78c verbatim (docker run -d --name qwen38-flash-orca-uncensored-bf16-mtp --restart no --label ai.yslab.recipe=qwen38-flash-next-bf16-ple --gpus all --ipc=host --shm-size 16g -p 127.0.0.1:18300:8000; loopback-only by default, options.port 18300 default, options.bind 127.0.0.1). Qualified = native 262,144 ctx, 8 max sequences, 0.80 gpu-mem util, PIECEWISE capture w/ explicit splitting_ops list, BF16 KV, no PLE prewarm, MTP=2 (RuntimeOptions defaults; --unsafe-override escapes the guard). The runtime unloads an existing recipe-labelled container before a guarded replacement. Deep MTP (depth>=5) would add --block-size 48 (QSA ring 12-token capacity x 48-token kernel alignment) - NOT used in the qualified profile. Thinking sampler: temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0 / presence 0.0 / repetition 1.0, medium reasoning effort (request-side, not baked into weights). Requires >=100 GiB free disk gate + HF account that accepted the Orcarouter gated-access conditions (browser step - a read token cannot grant access until accepted). Inferact @ 103a7608316173ca6edd49929544244de7ffda70 and RadixArk hybrid-BF16 tooling-compatible but runtime UNVALIDATED (COMPATIBILITY.md matrix).'

Use this recipe with your agent

Paste this into any agent chat — it fetches and applies the recipe:

Fetch https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

Or fetch it directly:

curl -fsSL https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2.json

Engine

vllm · source↗ · 9492b78c↗ · vllm/vllm-openai:qwen38-flash-next@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8↗

Install note: docker build -t qwen38-flash-dgx . per repo Dockerfile @ 9492b78c: FROM vllm/vllm-openai:qwen38-flash-next @ sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 (multi-arch, arm64 for the Spark's Grace CPU) + COPY vllm_ple_mmap.py + 5 patch scripts applied to the in-image vLLM sources (config, quantized lm_head + MTP, parallel lm_head linear attrs, MTP compressed-ignore, MTP load guard) + ple_layer.py hook appended (python3 -c ast.parse guard over all 5 patched files in the build); image label ai.yslab.vllm.version=0.1.dev20073+g8e685d198. Serving driven by the recipe CLI (scripts/qwen38-dgx-spark -> python -m recipe.cli): login (stores token in recipe cache), prepare orca-uncensored-bf16-mtp (downloads pinned checkpoint + only the 3 pinned source shards holding the 31 MTP tensors, writes audited local overlay; pre-requests config.json so gated-access failures happen early), serve orca-uncensored-bf16-mtp --mtp 2.

Model

Qwen3.8-Flash-Next-Uncensored-NVFP4-BF16PLE (nvfp4) · orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4↗
AA intelligence 40 (Artificial Analysis · measured)

Hardware

NVIDIA DGX Spark 128GB ×1 (Blackwell)

Context

max context: 256K tokens

Performance

Speed by context

Context	Conc.	Prefill	Decode	TTFT
32,768	1	—	44.2 tok/s	1324 ms	reported
240,051	1	2055.2 tok/s	—	116800 ms	reported

prefill = prompt processing · decode = generation. All decode/prefill figures are shown as reported.

reported

notes	Author-measured (docs/BENCHMARKS.md + results/orcarouter/*.json @ 9492b78c, one DGX Spark, Orcarouter checkpoint + BF16 MTP overlay ONLY): single-stream MTP depth sweep (32,768-token profile, concurrency 1, one max sequence, 0.80 util, 128-token warm-up, 3 fixed 256-token streamed samples; primary rate = ALL completion tokens incl. hidden reasoning / wall time) - MTP0 27.26, MTP1 33.17 (65.9%), MTP2 44.23 (73.7% = 532/722), MTP3 40.95 (56.7%), MTP4 41.84 (53.2%), MTP5 35.45 (29.1%), MTP6 35.71 (

notes

Author-measured (docs/BENCHMARKS.md + results/orcarouter/*.json @ 9492b78c, one DGX Spark, Orcarouter checkpoint + BF16 MTP overlay ONLY): single-stream MTP depth sweep (32,768-token profile, concurrency 1, one max sequence, 0.80 util, 128-token warm-up, 3 fixed 256-token streamed samples; primary rate = ALL completion tokens incl. hidden reasoning / wall time) - MTP0 27.26, MTP1 33.17 (65.9%), MTP2 44.23 (73.7% = 532/722), MTP3 40.95 (56.7%), MTP4 41.84 (53.2%), MTP5 35.45 (29.1%), MTP6 35.71 (

Install requirements

Docker (or a compatible OCI container runtime)
NVIDIA Container Toolkit (nvidia-container-toolkit) - exposes host GPUs to the container

NVIDIA driver floor: unknown - check the image tag

Source: NVIDIA CUDA Toolkit Release Notes - driver compatibility tables (https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/, fetched 2026-10-03).

Map the placeholders to your rig

Hardware-bound values in this recipe (device counts, bind-mount paths, utilization, ports) describe the author's machine, not yours - map them to your own rig before launching. Keep the engine flags, model, quant, and parallelism strategy; adjust paths and ports to the hardware you actually have.
GPU identity: list your GPUs with `nvidia-smi -L` and substitute your own GPU UUIDs/indices wherever the recipe pins the author's.
Model directory: bind-mount paths are author-side - point them at your own mount where the model files live.
Host port: pick a free port on your machine and publish the container there; keep the in-container port unchanged.

Downloads

Weights https://huggingface.co/orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4
revision 3a3b63161c0745390e5270179af42e46efc70799 (GATED repo - accept access conditions in browser first; direct BF16 PLE, ~95.37 GiB table; zero MTP tensors despite config advertising an MTP layer)
Drafter https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4
BF16 MTP overlay SOURCE: exactly 31 native BF16 MTP tensors from pinned revision 7b719225242aacd3dbd3f9407468c2ee9a9d2594 -> 4.86 GiB compact overlay built+audited by the recipe (sha256-per-shard, hash+dtype+name+arch+load-count guards, 31/31 load-guard ve
Engine image https://hub.docker.com/r/vllm/vllm-openai
vllm/vllm-openai:qwen38-flash-next @ sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8 + repo patch set (vllm_ple_mmap hook + 5 patch scripts); local tag qwen38-flash-dgx
Reference https://github.com/YSLAB-ai/Qwen3.8-Flash-Next-NVFP4-BF16PLE-DGX-Spark
9492b78c6bc04638587b42f2ffeb0680b0fcb3ef - src/recipe (cli/runtime/manifest/download/audit), Dockerfile + 5 patch scripts + vllm_ple_mmap.py, compatibility.json, results/orcarouter/*.json (mtp sweep, native-240k, concurrency, functional, readiness, summary

Revision notes

Ingested from GitHub repo YSLAB-ai/Qwen3.8-Flash-Next-NVFP4-BF16PLE-DGX-Spark @ 9492b78c6bc04638587b42f2ffeb0680b0fcb3ef (wave 3, kanban t_21a3a083). Same dgx-spark hardware family as bemlerlabs/Felliks (ONE profile created for all three, per card dedupe instruction) but DIFFERENT engine lane: this is the vLLM one (official qwen38-flash-next image) vs their SGLang. Distinct checkpoint row: orcarouter Uncensored-NVFP4 (gated) vs the site's existing RadixArk NVFP4 row. Recipe code Apache-2.0; weights under their own licenses/access terms (NOTICE).

Revision history

4ab2213ca7 recipe: update qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2 (r2: /max_context_len) 10 d ago diff
4f113d3723 recipe: create qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2 23 d ago diff
8717df2959 wave-3 fix: remove same-run mislinked dgx-spark ingests (hw.card resolved to legacy silent-created profile; re-ingesting with corrected card stem) t_21a3a083 23 d ago diff
94a7fed739 recipe: create qwen3-8-flash-next-uncensored-nvfp4-bf16-ple-dgx-spark-vllm-mtp2 23 d ago

Fork

Export

recipe.yaml (raw)