diff --git a/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml b/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml index 62cf20c71..bf27a41bc 100644 --- a/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml +++ b/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml @@ -70,7 +70,7 @@ config: runtime: container: 'docker run -d --name glm53-flash --user "$(id -u):$(id -g)" --workdir /cache --gpus all --shm-size 16g -p 127.0.0.1:8000:8000 --env-file docker/container.env -v serve.sh:/recipe/serve.sh:ro -v models/GLM-5.3-Flash-W4A16-MTP:/models/GLM-5.3-Flash-W4A16-MTP:ro -v models/GLM-5.3-Flash-DFlash2:/models/GLM-5.3-Flash-DFlash2:ro -v cache:/cache --entrypoint /bin/bash ghcr.io/morrowmake/vllm-cmp170hx@sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a /recipe/serve.sh' env: 'LAYOUT=tp4 (x16 links) or LAYOUT=pp4 (x4 links; 2312 max-num-batched-tokens). container.env: VLLM_ALLOW_PCIE_P2P_CUSTOM_ALLREDUCE=0 (set 1 if nvidia-smi topo -p2p r says OK for every pair; worth +5% decode/+13.7% prefill under TP4). HF_HOME/FLASHINFER_WORKSPACE_BASE/TORCHINDUCTOR_CACHE_DIR under ./cache so first-boot FlashInfer compile (4-5 min) happens once. Updated to Morrowmake release 1.7.2 2026-10-06: image pin 1.7.2-ab60b723ad, digest sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a. ./start.sh update preserves .env and adopts the 1.7.2 defaults (RecoverSSM on at TP4, adaptive DFlash2 depth on, BOOT_CHECK=1) unless set explicitly; upstream ignores the old VLLM_GLM5_MARLIN_PREFILL_CUDA switch (never set here).' - notes: 'FIELD REPORT 2026-10-08 (site owner): users report the current Morrowmake release "going off the rails" (output coherence degrading) on 4x CMP 170HX rigs; several pin Morrowmake release 1.6.0 instead. If output coherence degrades on 1.7.x, try the 1.6.0 image tag (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) before debugging your rig. Upstream tracking: Morrowmake/glm53-flash-cmp170hx-recipe issues #4 and #7/#8 (1.7.0 output-corruption fixes landed through 1.7.2). Remove this note once upstream confirms resolution. PP4 layout: activations-only inter-card traffic, tuned for x4-link cards, 2.34x faster prefill (6254 vs 2670), 2.01x KV pool. start.sh checks link width and suggests LAYOUT=pp4 automatically. The image runs at native-install speed (within 0.4%). All sm_80 feature flags are kill switches via env (VLLM_GLM5_PREFILL_KERNELS, VLLM_GLM5_DECODE_KERNELS, FAIR_PREFILL etc.).' + notes: 'FIELD REPORT UPDATE 2026-10-13 (site owner): 1.7.2 (image 1.7.2-ab60b723ad) is now reported working fine on 4x CMP 170HX rigs — upstream issues #7/#8 (TP4 long-gen context loss / endless repetition) and #4 (prefill scratch OOM) are confirmed fixed in 1.7.2. KEEPING the 1.6.0 image (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) as a known-good fallback: if output coherence degrades on your rig, trying 1.6.0 before debugging hardware remains reasonable. The old 2026-10-08 "off the rails" reports are resolved as of 1.7.2. PP4 layout: activations-only inter-card traffic, tuned for x4-link cards, 2.34x faster prefill (6254 vs 2670), 2.01x KV pool. start.sh checks link width and suggests LAYOUT=pp4 automatically. The image runs at native-install speed (within 0.4%). All sm_80 feature flags are kill switches via env (VLLM_GLM5_PREFILL_KERNELS, VLLM_GLM5_DECODE_KERNELS, FAIR_PREFILL etc.).' performance: claimed_by: submitter metrics: @@ -93,7 +93,7 @@ artifacts: kind: engine_image pointer_type: docker_digest pointer: 'https://github.com/Morrowmake/vllm-cmp170hx' - pin: 'image ghcr.io/morrowmake/vllm-cmp170hx:1.7.2-ab60b723ad, digest sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a (recipe release 1.7.2). Public image tags: 1.7.2-ab60b723ad, 1.6.0-3a2bf16dae, 1.5.0-e77f89da20. The 1.7.2 serve.sh is launch-compatible with this as-run PP4 config (same dflash k=3 path); the perf numbers below remain release-1.4.1 measurements (1.7.2 validates speeds unchanged within noise).' + pin: 'image ghcr.io/morrowmake/vllm-cmp170hx:1.7.2-ab60b723ad, digest sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a (recipe release 1.7.2). Public image tags: 1.7.2-ab60b723ad, 1.6.0-3a2bf16dae, 1.5.0-e77f89da20. The 1.7.2 serve.sh is launch-compatible with this as-run PP4 config (same dflash k=3 path); the perf numbers below remain release-1.4.1 measurements (1.7.2 validates speeds unchanged within noise). (1.6.0-3a2bf16dae retained as a fallback tag per field reports; 1.7.2 reported working fine 2026-10-13.)' - kind: reference pointer_type: git_commit @@ -103,7 +103,7 @@ provenance: source_urls: - 'https://github.com/Morrowmake/vllm-cmp170hx/tree/ampere-glm53' - 'https://huggingface.co/zai-org/GLM-5.3-Flash' - notes: 'Recipe by Morrowmake (github.com/Morrowmake/glm53-flash-cmp170hx-recipe, MIT licence), release 1.4.1 2026-09-27. Mirrored into localairecipes from the local checkout with every pin transcribed verbatim; engine+image+weights are digest/commit-pinned so this exact stack is reproducible. Updated 2026-10-06 from upstream release 1.4.1 to 1.7.2 (engine ab60b723ada, image 1.7.2-ab60b723ad): TP4 long-gen state recovery (#7), prefill scratch OOM (#9), cached-prefix garbage output (#10), Marlin prefill dependency order (#4), startup retry for slow warmup (#5); as-run launch args unchanged (still the 1.7.2 PP4 defaults); measured tok/s on site unchanged (upstream validates speeds unchanged within noise). FIELD REPORT 2026-10-08 (site owner): users report the current Morrowmake release "going off the rails" (output coherence degrading) on 4x CMP 170HX rigs; several pin Morrowmake release 1.6.0 instead. If output coherence degrades on 1.7.x, try the 1.6.0 image tag (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) before debugging your rig. Upstream tracking: Morrowmake/glm53-flash-cmp170hx-recipe issues #4 and #7/#8 (1.7.0 output-corruption fixes landed through 1.7.2). Remove this note once upstream confirms resolution.' + notes: 'Recipe by Morrowmake (github.com/Morrowmake/glm53-flash-cmp170hx-recipe, MIT licence), release 1.4.1 2026-09-27. Mirrored into localairecipes from the local checkout with every pin transcribed verbatim; engine+image+weights are digest/commit-pinned so this exact stack is reproducible. Updated 2026-10-06 from upstream release 1.4.1 to 1.7.2 (engine ab60b723ada, image 1.7.2-ab60b723ad): TP4 long-gen state recovery (#7), prefill scratch OOM (#9), cached-prefix garbage output (#10), Marlin prefill dependency order (#4), startup retry for slow warmup (#5); as-run launch args unchanged (still the 1.7.2 PP4 defaults); measured tok/s on site unchanged (upstream validates speeds unchanged within noise). FIELD REPORT UPDATE 2026-10-13 (site owner): 1.7.2 (image 1.7.2-ab60b723ad) is now reported working fine on 4x CMP 170HX rigs — upstream issues #7/#8 (TP4 long-gen context loss / endless repetition) and #4 (prefill scratch OOM) are confirmed fixed in 1.7.2. KEEPING the 1.6.0 image (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) as a known-good fallback: if output coherence degrades on your rig, trying 1.6.0 before debugging hardware remains reasonable. The old 2026-10-08 "off the rails" reports are resolved as of 1.7.2.' credit: - name: Morrowmake