diff --git a/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml b/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml index 35374c9b4..d1683fff1 100644 --- a/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml +++ b/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml @@ -70,7 +70,7 @@ config: runtime: container: 'docker run -d --name glm53-flash --user "$(id -u):$(id -g)" --workdir /cache --gpus all --shm-size 16g -p 127.0.0.1:8000:8000 --env-file docker/container.env -v serve.sh:/recipe/serve.sh:ro -v models/GLM-5.3-Flash-W4A16-MTP:/models/GLM-5.3-Flash-W4A16-MTP:ro -v models/GLM-5.3-Flash-DFlash2:/models/GLM-5.3-Flash-DFlash2:ro -v cache:/cache --entrypoint /bin/bash ghcr.io/morrowmake/vllm-cmp170hx@sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a /recipe/serve.sh' env: 'LAYOUT=tp4 (x16 links) or LAYOUT=pp4 (x4 links; 2312 max-num-batched-tokens). container.env: VLLM_ALLOW_PCIE_P2P_CUSTOM_ALLREDUCE=0 (set 1 if nvidia-smi topo -p2p r says OK for every pair; worth +5% decode/+13.7% prefill under TP4). HF_HOME/FLASHINFER_WORKSPACE_BASE/TORCHINDUCTOR_CACHE_DIR under ./cache so first-boot FlashInfer compile (4-5 min) happens once. Updated to Morrowmake release 1.7.2 2026-10-06: image pin 1.7.2-ab60b723ad, digest sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a. ./start.sh update preserves .env and adopts the 1.7.2 defaults (RecoverSSM on at TP4, adaptive DFlash2 depth on, BOOT_CHECK=1) unless set explicitly; upstream ignores the old VLLM_GLM5_MARLIN_PREFILL_CUDA switch (never set here).' - notes: 'PP4 layout: activations-only inter-card traffic, tuned for x4-link cards, 2.34x faster prefill (6254 vs 2670), 2.01x KV pool. start.sh checks link width and suggests LAYOUT=pp4 automatically. The image runs at native-install speed (within 0.4%). All sm_80 feature flags are kill switches via env (VLLM_GLM5_PREFILL_KERNELS, VLLM_GLM5_DECODE_KERNELS, FAIR_PREFILL etc.).' + notes: 'FIELD REPORT 2026-10-08 (site owner): users report the current Morrowmake release "going off the rails" (output coherence degrading) on 4x CMP 170HX rigs; several pin Morrowmake release 1.6.0 instead. If output coherence degrades on 1.7.x, try the 1.6.0 image tag (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) before debugging your rig. Upstream tracking: Morrowmake/glm53-flash-cmp170hx-recipe issues #4 and #7/#8 (1.7.0 output-corruption fixes landed through 1.7.2). Remove this note once upstream confirms resolution. PP4 layout: activations-only inter-card traffic, tuned for x4-link cards, 2.34x faster prefill (6254 vs 2670), 2.01x KV pool. start.sh checks link width and suggests LAYOUT=pp4 automatically. The image runs at native-install speed (within 0.4%). All sm_80 feature flags are kill switches via env (VLLM_GLM5_PREFILL_KERNELS, VLLM_GLM5_DECODE_KERNELS, FAIR_PREFILL etc.).' performance: claimed_by: submitter metrics: @@ -103,7 +103,7 @@ provenance: source_urls: - 'https://github.com/Morrowmake/vllm-cmp170hx/tree/ampere-glm53' - 'https://huggingface.co/zai-org/GLM-5.3-Flash' - notes: 'Recipe by Morrowmake (github.com/Morrowmake/glm53-flash-cmp170hx-recipe, MIT licence), release 1.4.1 2026-09-27. Mirrored into localairecipes from the local checkout with every pin transcribed verbatim; engine+image+weights are digest/commit-pinned so this exact stack is reproducible. Updated 2026-10-06 from upstream release 1.4.1 to 1.7.2 (engine ab60b723ada, image 1.7.2-ab60b723ad): TP4 long-gen state recovery (#7), prefill scratch OOM (#9), cached-prefix garbage output (#10), Marlin prefill dependency order (#4), startup retry for slow warmup (#5); as-run launch args unchanged (still the 1.7.2 PP4 defaults); measured tok/s on site unchanged (upstream validates speeds unchanged within noise).' + notes: 'Recipe by Morrowmake (github.com/Morrowmake/glm53-flash-cmp170hx-recipe, MIT licence), release 1.4.1 2026-09-27. Mirrored into localairecipes from the local checkout with every pin transcribed verbatim; engine+image+weights are digest/commit-pinned so this exact stack is reproducible. Updated 2026-10-06 from upstream release 1.4.1 to 1.7.2 (engine ab60b723ada, image 1.7.2-ab60b723ad): TP4 long-gen state recovery (#7), prefill scratch OOM (#9), cached-prefix garbage output (#10), Marlin prefill dependency order (#4), startup retry for slow warmup (#5); as-run launch args unchanged (still the 1.7.2 PP4 defaults); measured tok/s on site unchanged (upstream validates speeds unchanged within noise). FIELD REPORT 2026-10-08 (site owner): users report the current Morrowmake release "going off the rails" (output coherence degrading) on 4x CMP 170HX rigs; several pin Morrowmake release 1.6.0 instead. If output coherence degrades on 1.7.x, try the 1.6.0 image tag (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) before debugging your rig. Upstream tracking: Morrowmake/glm53-flash-cmp170hx-recipe issues #4 and #7/#8 (1.7.0 output-corruption fixes landed through 1.7.2). Remove this note once upstream confirms resolution.' credit: - name: Morrowmake