diff --git a/recipes/qwen3-8-flash-next-w4a16-autoround-dgx-spark-vllm-mtp4-iter7g/recipe.yaml b/recipes/qwen3-8-flash-next-w4a16-autoround-dgx-spark-vllm-mtp4-iter7g/recipe.yaml index 5b9f6c3ca..e6a2cfdf4 100644 --- a/recipes/qwen3-8-flash-next-w4a16-autoround-dgx-spark-vllm-mtp4-iter7g/recipe.yaml +++ b/recipes/qwen3-8-flash-next-w4a16-autoround-dgx-spark-vllm-mtp4-iter7g/recipe.yaml @@ -6,7 +6,7 @@ engine: source_type: fork git_url: 'https://github.com/kyaky/qwen3.8-flash-next-gb10-ultrafast' pinned_commit: 2ddeb883fe692e3eee1840655a3df7de8eddf87d - image_digest: 'https://hub.docker.com/r/vllm/vllm-openai@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8' + image_digest: 'docker.io/vllm/vllm-openai@sha256:fc120ece0a388cc0aa1caad4a9f1cd92113484ab7ec2fd0efadd62585be05bf8' install_note: 'Rootless podman 4.x/5 on ARM64 GB10 with NVIDIA CDI; CUDA 13.0-pinned driver. Build chain: recipe/build/image/build.sh (iter6c->iter6d) then build-iter7g.sh on top, then recipe/build/model/build.sh T80 dense-MTP g32 drafter directory (about 4.8 GiB). Measured-line image tag qwen38-flash-dgx:iter7g-20261015; the promoted v20-fp8 serving line runs the local podman tag qwen38-flash-dgx:fp8kv-promo-20261024 (fp8 KV read path + #59528 null-block backport; not built by the repo build chain). launch.sh enforces a fixed image tag and inspects the image via podman. Deterministic floor needs the env knobs MOE_ALIGN_STABLESORT=1, MARLIN_ATOMIC_ADD=0, TRITON_FORCE_FIRST_CONFIG=1 + the persistent ~/.cache/qwen38-compile mount, all sourced from recipe/config/v20-fp8/env (measured line: recipe/config/v19a-w5/env) under a cleared environment by launch.sh.' summary: 'Recipe by kyaky. Qwen3.8-Flash-Next W4A16-AutoRound hybrid (INT4 experts + FP8 side layers + INT8 head; 51B-token n-gram PLE table on NVMe mmap) on one DGX Spark (GB10, 121 GB unified memory) on a verified-deterministic vLLM floor (image qwen38-flash-dgx:iter7g-20261015 on the digest-pinned vllm/vllm-openai:qwen38-flash-next base): MoE align arrival-order shim, drafter-Marlin atomicAdd fix, pinned triton config, compile-cache mount. Measured line v19a-w5 = MTP=4 window-5 block verification: 76.8 tok/s single-stream, 261.2 tok/s aggregate at 8 streams (+9.2% N1 / +4.8% N8 over MTP=3, two boots per arm, same image). Promoted serving line v20-fp8: fp8 KV cache on a 24 GB pool at native 262,144 context, 1,291,120-token capacity. Chinese +31.3% via a 25,862-id (39.5%) Han draft-vocab rebalance (prose-zhen 34.8 to 45.7 tok/s). Stochastic-correctness gate 37/37 vs bit-exact fp64 oracle; per-round raw data + scripts ship in-repo; copy-agentic-1 failure disclosed.' license: CC-BY-4.0