{"slug":"qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-mtp-213-262k","title":"Qwen3.8-Flash-Next NVFP4 DGX Spark GB10 SGLang MTP-213 262K","summary":"Native-262K-context DGX Spark recipe for RadixArk/Qwen3.8-Flash-Next-NVFP4, validated on ASUS Ascent GX10 (same GB10 / 128GB unified memory as NVIDIA DGX Spark): the NVFP4 checkpoint is 135.24 GB and stock PLE host offload still lands the whole 51.2 GB table in host memory - but host memory and VRAM are the SAME 128 GB pool on GB10 - so this recipe adds two source-hash-guarded patches to a pinned SGLang image: (1) verified raw FP8 mmap keeps the full PLE table on NVMe with a bounded 256 MiB four-way pinned row cache for decode lookups (large prefill lookups bypass it), (2) QSA decode uses a PyTorch SDPA fallback on SM121 because the pinned FlashAttention-4 varlen path does not compile on GB10. prepare step converts the checkpoint's 128 FP8 PLE shards into a contiguous 51.2 GB mmap and byte-verifies 256 deterministic rows against the safetensors source. Official SGLang MTP-213 profile (2 speculative steps / top-k 1 / 3 draft tokens). Author-measured 2026-08-26 (benchmarks/asus-ascent-gx","license":"CC-BY-4.0","tier":"community","beta":false,"revision":{"no":2,"commit_sha":"28fe705dac20c012daa230c6139390ac9c3f389a","content_hash":"c5a8a1d93d215c46ce05e7bbeb9d6c37773c356d","created_at":"2026-10-20 15:34:13.16029-06","notes":"Ingested from GitHub repo Felliks/qwen38-flash-next-one-dgx-spark @ 5187a06c6c44a894ab6df2c7f07e005e1f8e66b9 (wave 3, kanban t_21a3a083). Validated on ASUS Ascent GX10; NVIDIA DGX Spark confirmations requested by author (same GB10 silicon - profile dgx-spark covers both, exact variant in tooltip/notes). Apache-2.0 code; Qwen Community License 1.0 weights. Measurements are from the included pinned checkpoint and launch path, not estimates copied from another GPU (author's statement)."},"engine_source":{"type":"fork","git_url":"https://github.com/Felliks/qwen38-flash-next-one-dgx-spark","base_engine":"sglang","pip_package":null,"base_version":"image commit d91c3682 (qwen4 source 73a255206f)","image_digest":"lmsysorg/sglang@sha256:14ed582518584c5c830206b5318a2c2769e68229c3422e48a28b952b3a888bd4","install_note":"Fully pinned per repo 'Pinned components' table: base image lmsysorg/sglang @ sha256:14ed582518584c5c830206b5318a2c2769e68229c3422e48a28b952b3a888bd4 (image commit d91c3682b0b429e4c70df63cd57f819588ce29b0, qwen4 experimental source 73a255206f916366c8d26d4022f82ddfb0ab558d) + the repo's 2 source-hash-guarded patches (patch_qwen4_ple_disk_cache.py, patch_sm121_qsa_sdpa.py - the build FAILS if upstream files no longer match expected SHA-256) + prepare_ple.py mmap build w/ 256-row byte verification. Local image tag qwen38-flash-next-gx10:73a255-ple-disk-v3. Requires aarch64 + compute_cap 12.1 (run-spark.sh require_gb10 gate) and >=230GB free NVMe.","pinned_commit":"5187a06c6c44a894ab6df2c7f07e005e1f8e66b9"},"credit":[{"name":"Felliks","url":"https://github.com/Felliks/qwen38-flash-next-one-dgx-spark"},{"name":"sgl-project (SGLang + official MTP-213 profile)","url":"https://github.com/sgl-project/sglang"}],"author":{"name":"localai-recipes","isSite":false},"uploaded_by":"localai-recipes","config":{"launch":["sglang","serve","--model-path","RadixArk/Qwen3.8-Flash-Next-NVFP4","--revision","7b719225242aacd3dbd3f9407468c2ee9a9d2594","--served-model-name","qwen38-flash-next-nvfp4-mtp","--trust-remote-code","--host","0.0.0.0","--port","30000","--quantization","modelopt_fp4","--fp4-gemm-backend","flashinfer_cutlass","--page-size","64","--mamba-radix-cache-strategy","extra_buffer","--mamba-track-interval","64","--max-mamba-cache-size","20","--mamba-ssm-dtype","float32","--chunked-prefill-size","4096","--max-running-requests","4","--max-total-tokens","524288","--context-length","262144","--mem-fraction-static","0.95","--allow-auto-truncate","--ple-offload-embedding","--reasoning-parser","qwen3","--tool-call-parser","qwen3_coder","--preferred-sampling-params","{\"temperature\":1.0,\"top_p\":0.95,\"top_k\":20,\"min_p\":0.0,\"presence_penalty\":0.0,\"repetition_penalty\":1.0}","--disable-prefill-cuda-graph","--cuda-graph-backend-decode","disabled","--disable-flashinfer-autotune","--speculative-algorithm","NEXTN","--speculative-num-steps","2","--speculative-eagle-topk","1","--speculative-num-draft-tokens","3"],"runtime":{"env":"SGLANG_QWEN4_PLE_DISK_CACHE_PATH=/ple-cache/ple-fp8.raw | SGLANG_QWEN4_PLE_DISK_CACHE_MANIFEST=/ple-cache/ple-fp8.raw.json | SGLANG_QWEN4_PLE_SOURCE_REVISION=7b719225242aacd3dbd3f9407468c2ee9a9d2594 | SGLANG_QWEN4_PLE_CACHE_BYTES=268435456 | SGLANG_QWEN4_PLE_CACHE_MAX_LOOKUP_ROWS=4096","notes":"Serve profile = run-spark.sh serve() @ 5187a06c verbatim (docker run -d --rm --init --gpus all --ipc host --shm-size 16g --memory 116g --memory-swap 116g --cpuset-cpus 0-19 -p 127.0.0.1:8000:30000; mounts HF cache + sglang cache + ple-cache ro; local image qwen38-flash-next-gx10:73a255-ple-disk-v3). MTP-213 = SGLang official profile for this model (lmsys.org blog 2026-08-26): 2 steps / top-k 1 / 3 draft. KV BF16 (SM121 eager QSA path rejects FP8 K/V - tested). Decode EAGER by design: disk-fed PLE rows cannot be captured as fixed host pointers in a CUDA graph; prefill graph disabled. Default sampling for thinking = temp 1.0/top_p 0.95/top_k 20/min_p 0.0; non-thinking per request = temp 0.7/top_p 0.80/top_k 20/presence_penalty 1.5 (Qwen model card). Overrides: API_PORT, QWEN_NEXT_DATA_DIR, CPUSET, BIND_ADDR (default 127.0.0.1 - do not expose unauthenticated endpoint publicly). Known limits: 4 active requests share the 524K pool (two full 262K contexts max); TP=1 only for disk-backed PLE."},"kv_pool_tokens":524288,"max_context_len":262144},"models":[{"slug":"qwen3-8-flash-next-nvfp4","name":"Qwen3.8-Flash-Next-NVFP4","hf_repo":"Qwen/Qwen3.8-Flash-Next","quant":"nvfp4"}],"hardware":[{"slug":"dgx-spark","vendor":"nvidia","card":"DGX Spark (GB10, unified memory)","card_family":"dgx-spark","vram_gb":128,"device_count":1}],"generation":"blackwell","tags":["asus-ascent-gx10","bf16-kv","dgx-spark","eager-decode","felliks","gb10","mtp-speculative","nvme-ple","sglang","sm121"],"tasks":["chat","code"],"aa":{"aa_index":40,"aa_provenance":"measured","aa_source_url":"https://artificialanalysis.ai/models/qwen3-8-flash-next","aa_index_version":null,"aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-06 18:30:00-06","aa_family_slug":"qwen3-8-flash-next","aa_resolved_slug":"qwen3-8-flash-next"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Author-measured 2026-08-26 (benchmarks/asus-ascent-gx10-2026-08-26.json @ 5187a06c, structured record): single_stream 512 generated tokens, runs 28.078/32.772/32.727 tok/s, steady_state 32.7 tok/s, steady TTFT 0.2857s; four_stream_batch 4x768 tokens x2 reps wall 32.90/33.03s -> aggregate 93.372/92.996 tok/s, per-request range 23.636-23.807 tok/s, observed scheduler peak 103.36 tok/s; resident system memory 120.45 GB / 128 GB incl. OS; GPU utilization ~90%; cold load ~9.5 min. Configuration: ctx\", \"tok_s\": 32.7000000000000028421709430404007434844970703125, \"points\": [{\"notes\": \"bench JSON @ 5187a06c: single-stream steady-state 32.7 tok/s (runs 28.08/32.77/32.73), TTFT 0.286s; MTP-213, BF16 KV\", \"ttft_ms\": 286, \"concurrency\": 1, \"context_len\": 262144, \"decode_tok_s\": 32.7000000000000028421709430404007434844970703125}, {\"notes\": \"bench JSON: four-stream aggregate 93.37/93.00 tok/s over 2 reps (scheduler peak 103.36); per-request 23.6-23.8 tok/s shared\", \"concurrency\": 4, \"context_len\": 262144, \"decode_tok_s\": 93.2000000000000028421709430404007434844970703125}], \"context_len\": 262144}"}],"perf_points":[{"id":1058,"context_len":262144,"prefill_tok_s":null,"decode_tok_s":32.7000000000000028421709430404007434844970703125,"ttft_ms":286,"concurrency":1,"claimed_by":"submitter","notes":"bench JSON @ 5187a06c: single-stream steady-state 32.7 tok/s (runs 28.08/32.77/32.73), TTFT 0.286s; MTP-213, BF16 KV"},{"id":1059,"context_len":262144,"prefill_tok_s":null,"decode_tok_s":93.2000000000000028421709430404007434844970703125,"ttft_ms":null,"concurrency":4,"claimed_by":"submitter","notes":"bench JSON: four-stream aggregate 93.37/93.00 tok/s over 2 reps (scheduler peak 103.36); per-request 23.6-23.8 tok/s shared"}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-mtp-213-262k.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-mtp-213-262k.json"},"requirements":null,"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4","pin":"revision 7b719225242aacd3dbd3f9407468c2ee9a9d2594 (135.24 GB NVFP4 checkpoint; 128 FP8 PLE shards -> contiguous 51.2 GB mmap via prepare_ple.py, 256-row byte verification)"},{"kind":"engine_image","pointer_type":"docker_digest","pointer":"https://hub.docker.com/r/lmsysorg/sglang","pin":"lmsysorg/sglang @ sha256:14ed582518584c5c830206b5318a2c2769e68229c3422e48a28b952b3a888bd4 (commit d91c3682b0b429e4c70df63cd57f819588ce29b0, qwen4 source 73a255206f916366c8d26d4022f82ddfb0ab558d) + 2 source-hash-guarded patches (PLE disk cache, SM121 QSA SD"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/Felliks/qwen38-flash-next-one-dgx-spark","pin":"5187a06c6c44a894ab6df2c7f07e005e1f8e66b9 - run-spark.sh (prepare/serve/smoke/status), Dockerfile, patch_qwen4_ple_disk_cache.py, patch_sm121_qsa_sdpa.py, prepare_ple.py, benchmarks/asus-ascent-gx10-2026-08-26.json, GitHub Actions validate workflow"}],"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-mtp-213-262k.json","alternate":"/r/qwen3-8-flash-next-nvfp4-dgx-spark-gb10-sglang-mtp-213-262k","describedby":"/openapi.json"},"updated_at":"2026-10-20 15:34:13.16029-06","created_at":"2026-10-07 11:30:10.929252-06","speeds":{"decode_tok_s":32.7000000000000028421709430404007434844970703125,"prefill_tok_s":0,"leaderboard":"/api/v1/leaderboard.json"}}