Qwen3.8-27B MXFP4 DFlash2 1x Radeon AI PRO R9700 WSL2 radiance

vllm serve /models/Qwen3.8-27B-MXFP4-official --served-model-name Qwen3.8-DFlash --host 0.0.0.0 --port 8080 --tensor-parallel-size 1 --language-model-only --gpu-memory-utilization 0.90 --max-model-len 220000 --max-num-seqs 3 --max-num-batched-tokens 2560 --kv-cache-dtype fp8 --mamba-cache-dtype bfloat16 --mamba-ssm-cache-dtype float16 --mamba-cache-mode align --enable-prefix-caching --attention-backend R4D --speculative-config '{"method":"dflash","model":"/models/tcclaviger/Qwen3.8-27B-DFlash2-FP8","num_speculative_tokens":7,"attention_backend":"TRITON_ATTN","draft_sample_method":"greedy"}' --no-async-scheduling --compilation-config '{"cudagraph_capture_sizes":[1,2,4,8,16,24,32,40],"pass_config":{"fuse_norm_quant":true,"fuse_act_quant":true}}' --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --override-generation-config '{"temperature":0.7,"top_p":0.95,"top_k":20}' --chat-template /patches/qwen-fixed-v22.3.jinja

env: 'Container wrapper (verbatim scripts/qwen38.sh cmd_start @ 3a15982): docker run -d --name qwen38-dflash --network host --ipc host --privileged --security-opt seccomp=unconfined --security-opt label=disable --cap-add SYS_PTRACE --shm-size 64m + WSL GPU passthrough (--device /dev/dxg, libdxcore.so + librocdxg.so + rocdxg dids.conf bind-mounted ro, HSA_ENABLE_DXG_DETECTION=1, render+video group-add) -v <radiance checkout>:/patches -v <libr4d build>:/r4d:ro -v scripts/entry.sh:/entry.sh:ro -v <models>:/models -v <caches>:/cache --env-file config/dflash.env -e VRAM_RESERVE_GIB(1.5; -Long: 1.0) -e VRAM_CAP(0.90; -Long: 0.95) --entrypoint bash <image> /entry.sh <launch argv>; entry.sh execs python3 /patches/available_memory.py which RECOMPUTES --gpu-memory-utilization from free VRAM (recorded 0.90 is the passed-in value). Load-bearing env (verbatim config/dflash.env @ HEAD): RADIANCE_FAST_DRAFT=1 RADIANCE_DRAFT_RERANK=80 RADIANCE_VERIFY_HEAD=1 RADIANCE_VERIFY_HEAD_MAX_M=32 RADIANCE_DRAFT_TAU=0.20 RADIANCE_DYNAMIC_WIDTH=1 (DYNW_ALPHA 0.35 / MARGIN 2 / MIN 2 / MIN_BATCH 3) RADIANCE_TP_PAD_DRAFTER=1 RADIANCE_MXFP4=1 RADIANCE_MXFP4_W4A8=1 (MIN_M=0, DECODE_NT=1, DECODE_MAX_M=64, EPIFAST=1, HOIST_QUANT=1, TRACED_QUANT=1, WPERM=1) RADIANCE_GDN_PATHS=both RADIANCE_GDN_MERGE_INPROJ=1 RADIANCE_GDN_FUSED_UPDATE=1 R4D_ATTN_FP8=3 RADIANCE_FP8_STREAM=1 RADIANCE_RMS_QUANT_FUSION=1 RADIANCE_SKINNY_GEMM=1 RADIANCE_WSL_DXG=1 VLLM_WSL2_ENABLE_PIN_MEMORY=1 HIP_VISIBLE_DEVICES=0 ROCR_VISIBLE_DEVICES=0 MXFP4_CUMODE=0 NCCL_PROTO=Simple VLLM_ROCM_USE_AITER=1 (MHA/MOE/MLA off, UNIFIED_ATTENTION on) VLLM_USE_V2_MODEL_RUNNER=1 RADIANCE_NVFP4_LMHEAD=bf16 RADIANCE_NVFP4_FP8_LAYERS=mxfp4 RADIANCE_NVFP4_BF16_LAYERS=in_proj_ba; caches TRITON_CACHE_DIR=/cache/triton TORCHINDUCTOR_CACHE_DIR=/cache/inductor VLLM_CACHE_ROOT=/cache/vllm AITER_ROOT_DIR=/cache/aiter HF_HUB_OFFLINE=1. Full measured set INCLUDING set-but-empty knobs (RADIANCE_DFLASH_CALIB=, RADIANCE_DFLASH_SELECTOR_TOPK=, RADIANCE_MXFP4_DECODE_BK= ...) lives in config/dflash.env @ the pinned sha — set-but-empty is deliberate, some knobs treat "" differently from unset.'
notes: "max-model-len -1 = context sized from free VRAM at each start by available_memory.py (~216k with default reserve 1.5 GiB / cap 0.90; ~260k via scripts/qwen38.sh start --long which sets 1.0 / 0.95; model limit 262144) — Windows desktop VRAM usage shifts the window between starts; insufficient free VRAM aborts startup (close GPU apps). Setup chain (scripts/qwen38.sh setup @ 3a15982): ROCDXG rocdxg-roct 1.2.2 deb (ROCm/librocdxg release); in-container GPU check asserts gfx1201 gcnArchName BEFORE large downloads; radiance checkout pinned 7d9a15a + patches/radiance-wsl-r9700.patch (adds available_memory.py + WSL dxg detection + pinned memory); models snapshot-pinned then scripts/fix_checkpoint.py applies a one-line config fix (vs upstream's 15-minute re-quantisation); libr4d b9e42ab + the radiance tree's r4d_radiance_extras_rx9.patch built inside the image and its r4d.so swapped over the image's own (stock predates the gated-delta-net overflow fix and NaNs this model); first start runs once to warm Triton/inductor caches (a cold start's autotuning buffers would cut the first window to ~100k). entry.sh applies the 20+ radiance patch chain (patch_quark_mxfp4, patch_wsl_dxg, patch_verify_head, patch_dynwidth, ...); the patch_qwen3_thinkoff step may no-op with a WARNING (thinking-off requests then return empty content). Server: OpenAI-compatible on :8080, model id Qwen3.8-DFlash; reasoning on by default (disable per request via chat_template_kwargs enable_thinking=false); tool calls via qwen3_coder parser + auto tool choice; defaults temp 0.7 / top_p 0.95 / top_k 20 (overridable); ONE request at a time (--max-num-seqs 1) — tuned for a single fast agent. Troubleshooting per README: WSL systemd needed for docker; decode dropping to ~10 tok/s = VRAM spilled to shared memory (stop, free VRAM, restart without -Long)."

Fetch https://localairecipes.com/api/v1/recipes/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance.json