diff --git a/recipes/qwen3-8-flash-next-w4a16-merlin-on-4x-rtx-3090-tp2xpp2-ep-mtp3-f/recipe.yaml b/recipes/qwen3-8-flash-next-w4a16-merlin-on-4x-rtx-3090-tp2xpp2-ep-mtp3-f/recipe.yaml index e00533e4a..8aefc67a1 100644 --- a/recipes/qwen3-8-flash-next-w4a16-merlin-on-4x-rtx-3090-tp2xpp2-ep-mtp3-f/recipe.yaml +++ b/recipes/qwen3-8-flash-next-w4a16-merlin-on-4x-rtx-3090-tp2xpp2-ep-mtp3-f/recipe.yaml @@ -4,7 +4,7 @@ engine: name: vllm version: 'v0.30.0 + 97 commits (Flash-Next v2.5.1 build)' source_type: fork - git_url: 'https://github.com/halt95/qwen38-flash-next-3090s' + git_url: 'https://github.com/halt95/merlin-engine' pinned_commit: 96a0d13b6e98ad4ef86e66a3f6ddb5dd9199a591 install_note: "Source build required — the vLLM fork ships as a release bundle, not an image (container image exists but is NOT rebuilt/GPU-served on v2.5.1 per README; the bare-metal route is the verified one). Build: clone the recipe repo @ 28cbb72, download release/v2.5.1/v2.5.1-from-upstream-v0.30.0.bundle.gz + build-artifacts-sm86-py313-cu130-v2.2.0.tar.gz from the repo's releases, then BUNDLE= ARTIFACTS= scripts/build-v2.5.sh ./vllm-v2.5 ./venv-v2.5 — the script verifies the base commit (vLLM v0.30.0 ced6857a), the fork commit 96a0d13b AND its tree hash 0df6d7ef, and hash-checks every build product before it serves. Needs python 3.13, CUDA 13.0+ driver (580+), ninja/C compiler for the first-serve FlashInfer/Triton JIT; compiled ops sm_86." summary: "Four-card RTX 3090 recipe for Qwen3.8-Flash-Next (125B MoE) in halt95's W4A16-Merlin build (Intel AutoRound INT4 + FP8 PLE + MTP draft, @ 17284bf): TP2 x PP2 with expert parallel, manual layer partition 25,23, FP8 KV with a 924,993-token pool (three full 262K sessions resident; block-size 4096, prefix-match-unit 64), MTP 3-token probabilistic draft with RecoverSSM, host-mapped PLE pull transport, host-resident embeddings, side-cache layout + Model Runner V2, 42 images/request. Agent-focused: 92.5% of follow-up prompt tokens from cache across 8 sessions, median follow-up first token 1.61 s (0.96 s solo). Maintainer-measured (submitter T0, v2.5.1 bench card 2026-10-06): 5,219-5,533 tok/s cold prefill 10K-261K, 158-174 tok/s single-stream decode (thinking on), 534 tok/s aggregate at 8 concurrent, 262K needle exact, GSM8K 198/200. Engine is a SOURCE BUILD: vLLM v0.30.0 + 97 commits (release bundle verified by commit AND tree hash); argv verbatim from scripts/serve-v2.5.sh @ 28cbb72." @@ -131,18 +131,18 @@ artifacts: - kind: engine_image pointer_type: git_commit - pointer: 'https://github.com/halt95/qwen38-flash-next-3090s' + pointer: 'https://github.com/halt95/merlin-engine' pin: "96a0d13b6e98ad4ef86e66a3f6ddb5dd9199a591 — the repo's vLLM fork: vLLM v0.30.0 + 97 commits (tree 0df6d7ef, verified by build script); source bundle sha256 d904e468…37d03d7c" - kind: reference pointer_type: git_commit - pointer: 'https://github.com/halt95/qwen38-flash-next-3090s' + pointer: 'https://github.com/halt95/merlin-engine' pin: '28cbb7293f134c244b443fb43bbf6e74ff4d8e24 — source recipe repo @ HEAD (scripts/serve-v2.5.sh is the verbatim argv source; scripts/build-v2.5.sh, scales/qsa_kv_scales_262k.json, release/, docker-compose.yml all at this commit)' provenance: credit: - name: halt95 - url: 'https://github.com/halt95/qwen38-flash-next-3090s' + url: 'https://github.com/halt95/merlin-engine' - name: 'Qwen (Qwen3.8-Flash-Next)' url: 'https://huggingface.co/Qwen/Qwen3.8-Flash-Next' @@ -159,9 +159,9 @@ provenance: name: 'peakcrosser7 (Flash-Next vLLM branch)' url: 'https://github.com/peakcrosser7/vllm/commits/release/qwen38next_offload' source_urls: - - 'https://github.com/halt95/qwen38-flash-next-3090s' - - 'https://github.com/halt95/qwen38-flash-next-3090s/blob/28cbb7293f134c244b443fb43bbf6e74ff4d8e24/README.md' - - 'https://github.com/halt95/qwen38-flash-next-3090s/blob/28cbb7293f134c244b443fb43bbf6e74ff4d8e24/benchmarks/2026-10-06/BENCH-CARD.md' + - 'https://github.com/halt95/merlin-engine' + - 'https://github.com/halt95/merlin-engine/blob/28cbb7293f134c244b443fb43bbf6e74ff4d8e24/README.md' + - 'https://github.com/halt95/merlin-engine/blob/28cbb7293f134c244b443fb43bbf6e74ff4d8e24/benchmarks/2026-10-06/BENCH-CARD.md' notes: "Community ingest of halt95/qwen38-flash-next-3090s @ 28cbb7293f134c244b443fb43bbf6e74ff4d8e24, release v2.5.1 (owner-named by Casey Oct 6; repo pushed 2026-10-06; Apache-2.0 + Qwen Community 1.0 for weights). Launch argv transcribed verbatim from scripts/serve-v2.5.sh @ 28cbb72 (selector-off legacy path, token-level checked against the raw script before POST); env exports verbatim from the same script. Perf numbers are the maintainer's 2026-10-06 bench card, logged claimed_by=submitter T0. Credits per the repo's own Credits section: Qwen (base), Intel AutoRound + RadixArk (quant components), DominikBucko (MTP packing), peakcrosser7 Flash-Next branch behind vLLM PRs #53896/#53899, aikitoria open-kernel-module P2P patch. Sibling repo halt95/Swift1.5-qwen38-flash-next-3090s is a post-train variant — NOT in scope this wave." max_context_len: 262144 kv_pool_tokens: 924993