diff --git a/recipes/qwen3-8-flash-next-fp8-4x-cmp-170hx-pp4-mtp-4-patched-vllm-g4bwy/recipe.yaml b/recipes/qwen3-8-flash-next-fp8-4x-cmp-170hx-pp4-mtp-4-patched-vllm-g4bwy/recipe.yaml index 859d12667..ab27af4dd 100644 --- a/recipes/qwen3-8-flash-next-fp8-4x-cmp-170hx-pp4-mtp-4-patched-vllm-g4bwy/recipe.yaml +++ b/recipes/qwen3-8-flash-next-fp8-4x-cmp-170hx-pp4-mtp-4-patched-vllm-g4bwy/recipe.yaml @@ -151,3 +151,4 @@ provenance: - 'https://huggingface.co/Qwen/Qwen3.8-Flash-Next-FP8' - 'https://github.com/amoghmunikote/cmpunlocker' notes: "Community ingest of g4bwy's recipe repo @ dcd5ba4 (pushed 2026-10-14, 'series refresh: patch 4 method-gated ... new replay tree 18e0ec8942'). Launch argv transcribed verbatim from serve-4x-1m.sh @ dcd5ba4; env exports are part of the script (VLLM_PP_LAYER_PARTITION, NCCL_P2P_DISABLE, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True, VLLM_SAFETENSORS_DROP_CACHE=1). Measured numbers are the author's benchmark/results.md tables: restored-silicon 4x-1M grid 2026-10-07 (74 SMs, bench-4x-1m-sm74-200w.json) + rebase certification on base 183625d4f9 2026-10-13 (logs/rebase-*.json) + decode step anatomy (212 tok/s stated as the single-stream roofline of this checkpoint on 4x170HX@200W at d4). Upstream (unpatched) vLLM refuses PP3+MTP+PLE on this checkpoint (drafter asserts on the last rank, PLE rejected across pipeline ranks, KV allocation dies with a bare StopIteration) - the 7 patches are load-bearing. Patch 5 (vocab-truncated MTP draft head, ~40k rows vs 248k) engages only when checkpoint/ artifacts exist and is worth ~9% single-stream; the int8-dense W8A16 lane (checkpoint/, patch 7) reads 216-226 tok/s (+15-20% over official FP8 at depth 4) with a 7.3% machine non-reproducibility floor measured and stated. Cards run the amoghmunikote/cmpunlocker module (SM compute + 64 GB HBM2e + Gen2 + BAR1); PCIe is Gen2 x16/x4 mixed, ~2 GB/s per pair, NODE distance; MTP acceptance 4.75-4.9 at depth 4 on pinned salt." +max_context_len: 1000000