diff --git a/recipes/qwen3-8-flash-next-nvfp4-1x6000-jj-mtp3/recipe.yaml b/recipes/qwen3-8-flash-next-nvfp4-1x6000-jj-mtp3/recipe.yaml index 948eebc6e..195f40503 100644 --- a/recipes/qwen3-8-flash-next-nvfp4-1x6000-jj-mtp3/recipe.yaml +++ b/recipes/qwen3-8-flash-next-nvfp4-1x6000-jj-mtp3/recipe.yaml @@ -6,7 +6,7 @@ engine: source_type: docker_image image_digest: 'localinferencelab/vllm:jovian-judgement-community-20260910-r34' git_url: 'https://github.com/local-inference-lab/rtx6kpro' -summary: 'Single-card Blackwell recipe for Qwen3.8-Flash-Next (hybrid GDN/QSA MoE, 48 layers, 512 experts) on one 96 GB RTX PRO 6000: the Community image community image R34 (localinferencelab/vllm, v0.26.1rc0+glm53.r34), TP1 with MTP3 speculative decoding (b12x MoE backend), FP8 KV cache, PLE n-gram table offloaded to host RAM (~27 GiB) so the 96 GB card fits 262,144-token context, modelopt NVFP4 checkpoint bind-mounted locally. Vision on (mm cache disabled), 64 seqs, 8192 batched tokens, util 0.98, FULL_AND_PIECEWISE cudagraphs, GLP-47 abliteration overlay alpha 1.0 with the projective steering hotfix. B12x PCIe allreduce stack pinned via env; measured 262K KV pool 619,559 tokens (2.36x admission), single-stream decode p50 ~117 tok/s (MTP acceptance 2.6/3), prompt chunks to ~2,900 tok/s p90. Weights: JMNI-Labs/Qwen3.8-Flash-Next-NVFP4-QAD5500-Hybrid — step-5500 QAD trained tensors (qad-step5500-ple1000 60215d26) recombined with main 7c4f1bc1 MXFP8 text-attention + W4A16 NVFP4 MTP experts; same-stack measured +5% MTP-normalized decode / +25% with MTP / slightly better prefill vs the main checkpoint (evals within noise, MMLU-Pro 66.9 vs 67.2).' +summary: 'Single-card Blackwell recipe for Qwen3.8-Flash-Next (hybrid GDN/QSA MoE, 48 layers, 512 experts) on one 96 GB RTX PRO 6000: the Community image community image R34 (localinferencelab/vllm, v0.26.1rc0+glm53.r34), TP1 with MTP3 speculative decoding (b12x MoE backend), 256k context. Benchmark-qualified on RTX PRO 6000 Blackwell Max-Q: decode 148 tok/s at ctx-0 (temp 1), prefill 3860 tok/s at 2048 batched tokens. Launch argv transcribed verbatim from the r35 community deployment. Weights: JMNI-Labs/Qwen3.8-Flash-Next-NVFP4-QAD5500-Hybrid — step-5500 QAD trained tensors recombined with main 7c4f1bc1 MXFP8 attention + NVFP4 MTP experts; same-stack measured +5% MTP-normalized decode, +25% with MTP vs the main checkpoint (evals within noise, MMLU-Pro 66.9 vs 67.2).' model: name: Qwen3.8-Flash-Next-NVFP4 family: qwen