{"slug":"qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance","title":"Qwen3.8-27B MXFP4 DFlash2 1x Radeon AI PRO R9700 WSL2 radiance","summary":"Qwen3.8-27B on a single AMD Radeon AI PRO R9700 (RDNA4 gfx1201, 32 GB) under Windows 11 + WSL2: vLLM radiance 0.9.3 fork pinned by image digest with AMD's official Quark AWQ-MXFP4 checkpoint, DFlash2 FP8 k=7 speculative drafter (59-63% acceptance, 5.1-5.4 tok per verify), hand-written W4A8 GEMM + libr4d FP8 paged-attention/GDN kernels pinned b9e42ab + rx9 patch, WSL pinned-memory fix, explicit context sizing, vision tower skipped. Author-measured 125-134 tok/s decode, 2750-2970 tok/s prefill at 1.9k. The author's defaults moved to a three-sequence long-context shape (max-num-seqs 3, batched tokens 2560, max-model-len 220000) backed by a KV-profiles pin the author labels REPORTED rather than measured, plus an opt-in KV cache subsystem (off|ram|disk) that leaves the serve command byte-identical when off. One-command PowerShell installer, OpenAI-compatible server on :8080.","license":"CC-BY-4.0","tier":"community","beta":false,"revision":{"no":2,"commit_sha":"99b06dc5a6b5393dde004bc60139fe19f74a1aeb","content_hash":"7b9f9a8cd306be8ff298ebb68ff1164fcf07a0ef","created_at":"2026-10-29 23:29:38.011666-06","notes":"Submitted 2026-10-09 by localai-recipes from lar-scout ingest card t_4a41485c (sweep QUEUE #2, 2026-10-09). Repo @ 3a159822ee98fe74e1478ccb835f921ccab0ab01 (HEAD via git ls-remote 2026-10-09; 2 commits, created 2026-10-08). Launch argv + docker flags + env token-verified verbatim against scripts/qwen38.sh (cmd_start/gpu_args), scripts/entry.sh, config/dflash.env and patches/radiance-wsl-r9700.patch at that sha. Pins carried: image digest sha256:45694209...fbc4c25 (tag 0.9.3), radiance commit 7d9a15a51d83733cc079fcf32ef4e3ec6b04ad7a, libr4d b9e42ab (+rx9 patch from the radiance tree), model 5233554c5fa56afda40150556b95573c2d7d29c0, drafter ee0cb26a8279b7910cc28d82a8a3e15e4728d56f, ROCDXG 1.2.2. Benchmarks are author-run WITH in-repo receipts (clean-install repro table + docs/BENCHMARKS.md + bench/bench.py shipped @ HEAD) \u2014 carried as submitter-T0 with receipts. First Radeon AI PRO R9700 recipe on the site (bucket empty, verified live 2026-10-09); first Windows/WSL2-native serving recipe; first radiance-lineage engine entry (base engine vllm, fork source). gfx1201-only kernels \u2014 other cards unsupported by design (container asserts gcnArchName before downloads). Dedupe: no other Qwen3.8-27B DFlash2-on-R9700 recipe on site; distinct from R9700 tooling-only repos (vllm-coherence, vllm-radiance-nix, r9700-stack). Hardware resolved to the pre-existing profile radeon-ai-pro-r9700 (seed-slug convention; no new profile created). UPDATE 2026-10-30 (repo @ 45da5063820c887d88c6d74af9215e5ba14bd7b3, 3 commits ahead of the pinned 7d9a15a5): the single-GPU profile defaults changed - serve-mxfp4.sh now defaults MAXSEQS=3 (\"3 is the long-context shape\"), CHUNK=2560 (was 4096, and the author states 2560 has NOT been benchmarked here the way 4096 was), and MAXLEN=220000 (was 65536, leaning on an explicit KV pin in kv-profiles.tsv). Those are the NEW defaults; the launch below records them explicitly so the recipe does not inherit them silently. Perf points below are the author's 2026-10-08 clean-install reproduction table measured at the PRE-CHANGE shape (4096 / 65536 / 1 seq) and were NOT re-measured at the new triple; the author labels the 220000 window as REPORTED (KV-profile pin), not measured, and CHUNK=2560 as unbenchmarked. New opt-in subsystem KVCACHE=off|ram|disk (off by default; the generated serve command is byte-identical when off) with KVCACHE_RAM_GIB / KVCACHE_DISK, 6 always-on behavioural patches for ram and a 15-patch instrumented set for disk, tooling in kv-cache/ (kvwatch.py, turnbench.py, equivbench.py, patch_*.py, kvcache-reap). New RADIANCE_GDN_LAZY=0 default, documented as corrupting multi-turn chat (repeat loops / empty replies from ~5 turns in) when enabled, and never applied at TP>=2. PAROQUANT.md gained MXFP6-PARO (W6A8) OCP MXFP6 E2M3 weights on fp8-WMMA kernels; PERFORMANCE.md gained TP=2 numbers from a second R9700 rig."},"engine_source":{"type":"fork","git_url":"https://codeberg.org/ggz14/radiance-vllm-mxfp4","base_engine":"vllm","pip_package":null,"base_version":"radiance 0.9.3 + ggz14/radiance-vllm-mxfp4 @ 45da506","image_digest":"docker.io/stilldeadcode/vllm-radiance@sha256:45694209177a55a1ab3ba6702fe6e978b1b66a6e66ae3fc066f8d579f7bc4c25","install_note":"NOT installable from the stock vLLM: serving runs the digest-pinned stilldeadcode/vllm-radiance:0.9.3 image; entry.sh applies the repo's patches/radiance-wsl-r9700.patch over the pinned radiance checkout, compiles the gfx1201 W4A8 GEMM with hipcc, swaps in the pinned libr4d r4d.so, then execs vLLM through available_memory.py (VRAM-budgeted context sizing). Requires WSL2 with /dev/dxg + ROCDXG 1.2.2; gfx1201-only by design.","pinned_commit":"45da5063820c887d88c6d74af9215e5ba14bd7b3"},"credit":[{"name":"mike2153 (recipe, WSL2 installer, benchmarks)","url":"https://github.com/mike2153/mbea-qwen38-dflash"},{"name":"StillDeadcode (vllm-radiance image + libr4d kernels)","url":"https://codeberg.org/StillDeadcode/libr4d"},{"name":"ggz14 (radiance-vllm-mxfp4 fork)","url":"https://codeberg.org/ggz14/radiance-vllm-mxfp4"},{"name":"magiccodingman (radiance RDNA4 kernel lineage)"},{"name":"Inco AI / tcclaviger (DFlash2 FP8 drafter requant)","url":"https://huggingface.co/tcclaviger/Qwen3.8-27B-DFlash2-FP8"},{"name":"AMD (official Qwen3.8-27B Quark AWQ-MXFP4 checkpoint + ROCm librocdxg)","url":"https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4"}],"author":{"name":"localai-recipes","isSite":false},"uploaded_by":"localai-recipes","config":{"launch":["vllm","serve","/models/Qwen3.8-27B-MXFP4-official","--served-model-name","Qwen3.8-DFlash","--host","0.0.0.0","--port","8080","--tensor-parallel-size","1","--language-model-only","--gpu-memory-utilization","0.90","--max-model-len","220000","--max-num-seqs","3","--max-num-batched-tokens","2560","--kv-cache-dtype","fp8","--mamba-cache-dtype","bfloat16","--mamba-ssm-cache-dtype","float16","--mamba-cache-mode","align","--enable-prefix-caching","--attention-backend","R4D","--speculative-config","{\"method\":\"dflash\",\"model\":\"/models/tcclaviger/Qwen3.8-27B-DFlash2-FP8\",\"num_speculative_tokens\":7,\"attention_backend\":\"TRITON_ATTN\",\"draft_sample_method\":\"greedy\"}","--no-async-scheduling","--compilation-config","{\"cudagraph_capture_sizes\":[1,2,4,8,16,24,32,40],\"pass_config\":{\"fuse_norm_quant\":true,\"fuse_act_quant\":true}}","--enable-auto-tool-choice","--tool-call-parser","qwen3_coder","--reasoning-parser","qwen3","--override-generation-config","{\"temperature\":0.7,\"top_p\":0.95,\"top_k\":20}","--chat-template","/patches/qwen-fixed-v22.3.jinja"],"runtime":{"env":"Container wrapper (verbatim scripts/qwen38.sh cmd_start @ 3a15982): docker run -d --name qwen38-dflash --network host --ipc host --privileged --security-opt seccomp=unconfined --security-opt label=disable --cap-add SYS_PTRACE --shm-size 64m + WSL GPU passthrough (--device /dev/dxg, libdxcore.so + librocdxg.so + rocdxg dids.conf bind-mounted ro, HSA_ENABLE_DXG_DETECTION=1, render+video group-add) -v <radiance checkout>:/patches -v <libr4d build>:/r4d:ro -v scripts/entry.sh:/entry.sh:ro -v <models>:/models -v <caches>:/cache --env-file config/dflash.env -e VRAM_RESERVE_GIB(1.5; -Long: 1.0) -e VRAM_CAP(0.90; -Long: 0.95) --entrypoint bash <image> /entry.sh <launch argv>; entry.sh execs python3 /patches/available_memory.py which RECOMPUTES --gpu-memory-utilization from free VRAM (recorded 0.90 is the passed-in value). Load-bearing env (verbatim config/dflash.env @ HEAD): RADIANCE_FAST_DRAFT=1 RADIANCE_DRAFT_RERANK=80 RADIANCE_VERIFY_HEAD=1 RADIANCE_VERIFY_HEAD_MAX_M=32 RADIANCE_DRAFT_TAU=0.20 RADIANCE_DYNAMIC_WIDTH=1 (DYNW_ALPHA 0.35 / MARGIN 2 / MIN 2 / MIN_BATCH 3) RADIANCE_TP_PAD_DRAFTER=1 RADIANCE_MXFP4=1 RADIANCE_MXFP4_W4A8=1 (MIN_M=0, DECODE_NT=1, DECODE_MAX_M=64, EPIFAST=1, HOIST_QUANT=1, TRACED_QUANT=1, WPERM=1) RADIANCE_GDN_PATHS=both RADIANCE_GDN_MERGE_INPROJ=1 RADIANCE_GDN_FUSED_UPDATE=1 R4D_ATTN_FP8=3 RADIANCE_FP8_STREAM=1 RADIANCE_RMS_QUANT_FUSION=1 RADIANCE_SKINNY_GEMM=1 RADIANCE_WSL_DXG=1 VLLM_WSL2_ENABLE_PIN_MEMORY=1 HIP_VISIBLE_DEVICES=0 ROCR_VISIBLE_DEVICES=0 MXFP4_CUMODE=0 NCCL_PROTO=Simple VLLM_ROCM_USE_AITER=1 (MHA/MOE/MLA off, UNIFIED_ATTENTION on) VLLM_USE_V2_MODEL_RUNNER=1 RADIANCE_NVFP4_LMHEAD=bf16 RADIANCE_NVFP4_FP8_LAYERS=mxfp4 RADIANCE_NVFP4_BF16_LAYERS=in_proj_ba; caches TRITON_CACHE_DIR=/cache/triton TORCHINDUCTOR_CACHE_DIR=/cache/inductor VLLM_CACHE_ROOT=/cache/vllm AITER_ROOT_DIR=/cache/aiter HF_HUB_OFFLINE=1. Full measured set INCLUDING set-but-empty knobs (RADIANCE_DFLASH_CALIB=, RADIANCE_DFLASH_SELECTOR_TOPK=, RADIANCE_MXFP4_DECODE_BK= ...) lives in config/dflash.env @ the pinned sha \u2014 set-but-empty is deliberate, some knobs treat \"\" differently from unset.","notes":"max-model-len -1 = context sized from free VRAM at each start by available_memory.py (~216k with default reserve 1.5 GiB / cap 0.90; ~260k via scripts/qwen38.sh start --long which sets 1.0 / 0.95; model limit 262144) \u2014 Windows desktop VRAM usage shifts the window between starts; insufficient free VRAM aborts startup (close GPU apps). Setup chain (scripts/qwen38.sh setup @ 3a15982): ROCDXG rocdxg-roct 1.2.2 deb (ROCm/librocdxg release); in-container GPU check asserts gfx1201 gcnArchName BEFORE large downloads; radiance checkout pinned 7d9a15a + patches/radiance-wsl-r9700.patch (adds available_memory.py + WSL dxg detection + pinned memory); models snapshot-pinned then scripts/fix_checkpoint.py applies a one-line config fix (vs upstream's 15-minute re-quantisation); libr4d b9e42ab + the radiance tree's r4d_radiance_extras_rx9.patch built inside the image and its r4d.so swapped over the image's own (stock predates the gated-delta-net overflow fix and NaNs this model); first start runs once to warm Triton/inductor caches (a cold start's autotuning buffers would cut the first window to ~100k). entry.sh applies the 20+ radiance patch chain (patch_quark_mxfp4, patch_wsl_dxg, patch_verify_head, patch_dynwidth, ...); the patch_qwen3_thinkoff step may no-op with a WARNING (thinking-off requests then return empty content). Server: OpenAI-compatible on :8080, model id Qwen3.8-DFlash; reasoning on by default (disable per request via chat_template_kwargs enable_thinking=false); tool calls via qwen3_coder parser + auto tool choice; defaults temp 0.7 / top_p 0.95 / top_k 20 (overridable); ONE request at a time (--max-num-seqs 1) \u2014 tuned for a single fast agent. Troubleshooting per README: WSL systemd needed for docker; decode dropping to ~10 tok/s = VRAM spilled to shared memory (stop, free VRAM, restart without -Long)."}},"models":[{"slug":"qwen3-8-27b-mxfp4","name":"Qwen3.8-27B-MXFP4","hf_repo":"amd/Qwen3.8-27B-Quark-AWQ-MXFP4","quant":"mxfp4-official"}],"hardware":[{"slug":"radeon-ai-pro-r9700","vendor":"amd","card":"Radeon AI Pro R9700","card_family":"radeon-ai-pro-r9700","vram_gb":32,"device_count":1}],"generation":"amd","tags":["amd","dflash2","gfx1201","long-context","mxfp4","one-command-installer","rdna4","speculative","w4a8","windows","wsl2"],"tasks":["chat","code"],"aa":{"aa_index":34,"aa_provenance":"published","aa_source_url":"https://artificialanalysis.ai/models/qwen3-8-27b","aa_index_version":null,"aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-07 18:00:00-06","aa_family_slug":"qwen-3-8-27b","aa_resolved_slug":"qwen-3-8-27b"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Author-measured on the shipped profile WITH in-repo receipts: clean-install reproduction table (README, 2026-10-08) + raw tables docs/BENCHMARKS.md + shipped bench/bench.py harness (.\\\\qwen38.ps1 bench) @ repo HEAD 3a15982. Decode greedy 125-134 tok/s (medians of 3 starts; clean repro 125.3); sampled (temp 0.7) 117-126. Prefill 2750-2970 tok/s @1.9k prompt (TTFT 0.66-0.70 s); 32k/98k/164k/258k prompts prefill in 10.9/41/83/164 s. Decode deep in context 165 @32k / 136 @98k / 145 @164k / 115 @258k. DFlash2 acceptance 59-63%, 5.1-5.4 tok per verify. Long-context recall 8/8 @ a 257,768-token prompt; coding 10/12 runs pass all 46 tests on a 1.9k Rust spec. Context ~216k default / ~260k -Long (model 262144, VRAM-sized per start). Load ~2.5-3 min. Reference: same model, tuned llama.cpp IQ4_XS build decodes 52-68 tok/s on this card (different prompt; rough per README). UPDATE 2026-10-30 (repo @ 45da5063820c887d88c6d74af9215e5ba14bd7b3, 3 commits ahead of the pinned 7d9a15a5): the single-GPU profile defaults changed - serve-mxfp4.sh now defaults MAXSEQS=3 (\\\"3 is the long-context shape\\\"), CHUNK=2560 (was 4096, and the author states 2560 has NOT been benchmarked here the way 4096 was), and MAXLEN=220000 (was 65536, leaning on an explicit KV pin in kv-profiles.tsv). Those are the NEW defaults; the launch below records them explicitly so the recipe does not inherit them silently. Perf points below are the author's 2026-10-08 clean-install reproduction table measured at the PRE-CHANGE shape (4096 / 65536 / 1 seq) and were NOT re-measured at the new triple; the author labels the 220000 window as REPORTED (KV-profile pin), not measured, and CHUNK=2560 as unbenchmarked. New opt-in subsystem KVCACHE=off|ram|disk (off by default; the generated serve command is byte-identical when off) with KVCACHE_RAM_GIB / KVCACHE_DISK, 6 always-on behavioural patches for ram and a 15-patch instrumented set for disk, tooling in kv-cache/ (kvwatch.py, turnbench.py, equivbench.py, patch_*.py, kvcache-reap). New RADIANCE_GDN_LAZY=0 default, documented as corrupting multi-turn chat (repeat loops / empty replies from ~5 turns in) when enabled, and never applied at TP>=2. PAROQUANT.md gained MXFP6-PARO (W6A8) OCP MXFP6 E2M3 weights on fp8-WMMA kernels; PERFORMANCE.md gained TP=2 numbers from a second R9700 rig.\", \"tok_s\": 125, \"context_len\": 216000}"}],"perf_points":[{"id":2215,"context_len":1920,"prefill_tok_s":2744,"decode_tok_s":125.2999999999999971578290569595992565155029296875,"ttft_ms":700,"concurrency":1,"claimed_by":"submitter","notes":"Coding task, greedy x3 median \u2014 clean-install reproduction table (README, 2026-10-08); published range 125-134 across 3 starts"},{"id":2216,"context_len":1921,"prefill_tok_s":2757,"decode_tok_s":116.7000000000000028421709430404007434844970703125,"ttft_ms":700,"concurrency":1,"claimed_by":"submitter","notes":"Coding task, sampled (temp 0.7, top-p 0.95) x3 median \u2014 clean-install reproduction table; published range 117-126"},{"id":2217,"context_len":34741,"prefill_tok_s":2762,"decode_tok_s":138.400000000000005684341886080801486968994140625,"ttft_ms":12580,"concurrency":1,"claimed_by":"submitter","notes":"~32k long prompt \u2014 clean-install reproduction table; docs/BENCHMARKS.md: 32k prefill 10.9 s, decode deep-in-context 165 tok/s at 32k"},{"id":2218,"context_len":98000,"prefill_tok_s":null,"decode_tok_s":136,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"Decode deep in context at 98k (docs/BENCHMARKS.md long-prompt arm; 98k prefill 41 s)"},{"id":2219,"context_len":164000,"prefill_tok_s":null,"decode_tok_s":145,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"Decode deep in context at 164k (docs/BENCHMARKS.md; 164k prefill 83 s)"},{"id":2220,"context_len":258000,"prefill_tok_s":null,"decode_tok_s":115,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"Decode deep in context at 258k (docs/BENCHMARKS.md; 258k prefill 164 s); long-context recall 8/8 at a 257,768-token prompt"}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance.json"},"requirements":null,"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4","pin":"5233554c5fa56afda40150556b95573c2d7d29c0 \u2014 AMD official Quark AWQ-MXFP4 checkpoint (scripts/qwen38.sh MODEL_REV; snapshot_download revision-pinned; scripts/fix_checkpoint.py one-line config fix)"},{"kind":"drafter","pointer_type":"hf_model","pointer":"https://huggingface.co/tcclaviger/Qwen3.8-27B-DFlash2-FP8","pin":"ee0cb26a8279b7910cc28d82a8a3e15e4728d56f \u2014 DFlash2 FP8 drafter, k=7 (scripts/qwen38.sh DRAFT_REV)"},{"kind":"engine_image","pointer_type":"docker_digest","pointer":"https://hub.docker.com/r/stilldeadcode/vllm-radiance","pin":"sha256:45694209177a55a1ab3ba6702fe6e978b1b66a6e66ae3fc066f8d579f7bc4c25 (tag 0.9.3; scripts/qwen38.sh IMAGE)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/mike2153/mbea-qwen38-dflash","pin":"3a159822ee98fe74e1478ccb835f921ccab0ab01 @ HEAD \u2014 scripts/qwen38.sh cmd_start is the verbatim argv/env source; config/dflash.env the measured env; patches/radiance-wsl-r9700.patch the local radiance overlay"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://codeberg.org/StillDeadcode/libr4d","pin":"b9e42ab (short-sha pin per scripts/qwen38.sh R4D_PIN; built with the radiance tree's r4d_radiance_extras_rx9.patch \u2014 stock libr4d NaNs this model's gated-delta-net layers)"}],"links":{"self":"/api/v1/recipes/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance.json","alternate":"/r/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance","describedby":"/openapi.json"},"updated_at":"2026-10-29 23:29:38.011666-06","created_at":"2026-10-08 20:07:53.461092-06","speeds":{"decode_tok_s":130.69999999999998863131622783839702606201171875,"prefill_tok_s":2762,"leaderboard":"/api/v1/leaderboard.json"}}