diff --git a/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml b/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml index 97087ca2f..2955e3087 100644 --- a/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml +++ b/recipes/glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141/recipe.yaml @@ -1,14 +1,14 @@ slug: glm-5-3-flash-4x170hx-tp4-pp4-morrowmake-141 -title: 'GLM-5.3-Flash on 4x CMP 170HX — Morrowmake recipe 1.7.2, TP4 DFlash2' +title: 'GLM-5.3-Flash on 4x CMP 170HX — Morrowmake recipe 1.8.0, TP4 DFlash2' engine: name: vllm - version: 'Morrowmake/vllm-cmp170hx @ ab60b723ada' + version: 'Morrowmake/vllm-cmp170hx @ 819146023f' source_type: docker_image - image_digest: 'ghcr.io/morrowmake/vllm-cmp170hx@sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a' - image: 'ghcr.io/morrowmake/vllm-cmp170hx:1.7.2-ab60b723ad' + image_digest: 'ghcr.io/morrowmake/vllm-cmp170hx@sha256:18a2f26207568128e1d792ce923e86a74e40def1e43e492db83132cab8cdbe14' + image: 'ghcr.io/morrowmake/vllm-cmp170hx:engine-819146023f' git_url: 'https://github.com/Morrowmake/vllm-cmp170hx' - pinned_commit: ab60b723ada254a442a4ba5ff27bf837aa27ef83 -summary: 'PP4 (pipeline-parallel 4, TP1) as-run on a 4x CMP 170HX rig 2026-09-28 at --gpu-memory-utilization 0.93 after 0.95 OOM-killed warmup (a 72 MiB KDA prefill workspace did not fit in the ~70 MiB left on the last card). Author 1.4.1 measurements below are TP4 unless labeled PP4; KV pool 2,320,328 tokens under PP4. Recipe release 1.7.2 2026-10-06 (engine ab60b723, image 1.7.2-ab60b723ad) — updated from 1.4.1; upstream fixes: TP4 long-gen state recovery (#7), prefill scratch OOM (#9), cached-prefix garbage output (#10), Marlin prefill dependency order (#4); measured tok/s on site unchanged (upstream validates speeds unchanged within noise).' + pinned_commit: 819146023facb31c04f1f8512e52464835a10d40 +summary: 'PP4 (pipeline-parallel 4, TP1) as-run on a 4x CMP 170HX rig 2026-09-28 at --gpu-memory-utilization 0.93 after 0.95 OOM-killed warmup (a 72 MiB KDA prefill workspace did not fit in the ~70 MiB left on the last card). Author 1.8.0 measurements below are TP4 unless labeled PP4; KV pool 2,088,960 tokens under PP4 (author-published 1.8.0 figure). Recipe release 1.8.0 2026-10-16 (engine 819146023f, image engine-819146023f) — updated from 1.7.2; engine rebased on upstream vLLM 4e0681ed56, same context/batch/fair-prefill defaults; 21 engine-switch env names renamed (old names silently ignored at code defaults); container-only install; author re-measured throughput/KV tables at 1.8.0.' model: name: GLM-5.3-Flash-W4A16-MTP family: glm @@ -51,6 +51,8 @@ config: - '8' - '--max-num-batched-tokens' - '2312' + - '--long-prefill-token-threshold' + - '0' - '--gpu-memory-utilization' - '0.93' - '--trust-remote-code' @@ -60,7 +62,7 @@ config: - '--tool-call-parser' - glm47 - '--speculative-config' - - '{"method":"dflash","model":"/models/GLM-5.3-Flash-DFlash2","num_speculative_tokens":3}' + - '{"method":"dflash","model":"/models/GLM-5.3-Flash-DFlash2","num_speculative_tokens":7,"adaptive_k":{"by_load":[7,5,3],"accept":{"costs":[1.0,1.0894,1.1375,1.2001,1.2239],"costs_multi":[1.0,1.0889,1.1194,1.2001,1.2239],"hysteresis":0.03},"log_interval":0}}' - '--block-size' - '4608' - '--prefill-chunk-with-decodes' @@ -68,9 +70,9 @@ config: - '--max-num-partial-prefills' - '2' runtime: - container: 'docker run -d --name glm53-flash --user "$(id -u):$(id -g)" --workdir /cache --gpus all --shm-size 16g -p 127.0.0.1:8000:8000 --env-file docker/container.env -v serve.sh:/recipe/serve.sh:ro -v models/GLM-5.3-Flash-W4A16-MTP:/models/GLM-5.3-Flash-W4A16-MTP:ro -v models/GLM-5.3-Flash-DFlash2:/models/GLM-5.3-Flash-DFlash2:ro -v cache:/cache --entrypoint /bin/bash ghcr.io/morrowmake/vllm-cmp170hx@sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a /recipe/serve.sh' - env: 'LAYOUT=tp4 (x16 links) or LAYOUT=pp4 (x4 links; 2312 max-num-batched-tokens). container.env: VLLM_ALLOW_PCIE_P2P_CUSTOM_ALLREDUCE=0 (set 1 if nvidia-smi topo -p2p r says OK for every pair; worth +5% decode/+13.7% prefill under TP4). HF_HOME/FLASHINFER_WORKSPACE_BASE/TORCHINDUCTOR_CACHE_DIR under ./cache so first-boot FlashInfer compile (4-5 min) happens once. Updated to Morrowmake release 1.7.2 2026-10-06: image pin 1.7.2-ab60b723ad, digest sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a. ./start.sh update preserves .env and adopts the 1.7.2 defaults (RecoverSSM on at TP4, adaptive DFlash2 depth on, BOOT_CHECK=1) unless set explicitly; upstream ignores the old VLLM_GLM5_MARLIN_PREFILL_CUDA switch (never set here).' - notes: 'FIELD REPORT UPDATE 2026-10-13 (site owner): 1.7.2 (image 1.7.2-ab60b723ad) is now reported working fine on 4x CMP 170HX rigs — upstream issues #7/#8 (TP4 long-gen context loss / endless repetition) and #4 (prefill scratch OOM) are confirmed fixed in 1.7.2. KEEPING the 1.6.0 image (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) as a known-good fallback: if output coherence degrades on your rig, trying 1.6.0 before debugging hardware remains reasonable. The old 2026-10-08 "off the rails" reports are resolved as of 1.7.2. PP4 layout: activations-only inter-card traffic, tuned for x4-link cards, 2.34x faster prefill (6254 vs 2670), 2.01x KV pool. start.sh checks link width and suggests LAYOUT=pp4 automatically. The image runs at native-install speed (within 0.4%). All sm_80 feature flags are kill switches via env (VLLM_GLM5_PREFILL_KERNELS, VLLM_GLM5_DECODE_KERNELS, FAIR_PREFILL etc.).' + container: 'docker run -d --name glm53-flash --user "$(id -u):$(id -g)" --workdir /cache --gpus all --shm-size 16g -p 127.0.0.1:8000:8000 --env-file docker/container.env -v serve.sh:/recipe/serve.sh:ro -v boot_check.py:/recipe/boot_check.py:ro -v p2p_check.py:/recipe/p2p_check.py:ro -v p2p_check.sh:/recipe/p2p_check.sh:ro -v p2p_probe.cu:/recipe/p2p_probe.cu:ro -v models/GLM-5.3-Flash-W4A16-MTP:/models/GLM-5.3-Flash-W4A16-MTP:ro -v models/GLM-5.3-Flash-DFlash2:/models/GLM-5.3-Flash-DFlash2:ro -v cache:/cache --entrypoint /bin/bash ghcr.io/morrowmake/vllm-cmp170hx@sha256:18a2f26207568128e1d792ce923e86a74e40def1e43e492db83132cab8cdbe14 /recipe/serve.sh (1.8.0: one-command start is ./start.sh or docker compose up — a prepare service makes ./cache and models writable by the checkout owner and downloads the checkpoints; the server runs as the checkout owner, never root; HOST_UID/HOST_GID override; first boot compiles kernel caches into ./cache and is slower, later boots reuse them)' + env: 'AS-RUN 2026-09-28 (unchanged): LAYOUT=pp4 (PP4/TP1), GPU_UTIL=0.93 — 0.95 filled the last 170HX to ~70 MiB free and a 72 MiB KDA prefill workspace OOM-killed warmup; 0.93 leaves ~1.3 GiB headroom the KV pool will not claim (deviation from serve.sh default GPU_UTIL=0.95). --host 0.0.0.0 is the as-run box value; the 1.8.0 default is HOST=127.0.0.1 (loopback) and the container publishes 127.0.0.1:8000 only. serve.sh v1.8.0 serving defaults behind this launch (documented here so a copy-paste into 1.8.0 keeps every feature): MAX_BATCHED=2312 (GLM_DEFAULT_MAX_BATCHED_PP4; TP4 default is 3463 = 3,456-token chunks + 7 draft slots), BLOCK_SIZE=4608 (PP4+dflash), FAIR_CHUNK=384 + FAIR_PARTIAL=2 (FAIR_PREFILL=1), PREFILL_CAP=0 -> --long-prefill-token-threshold 0 (keep 0; >0 costs -15.3% prefill / +16.5% TTFT on a 23K prompt and silently disables the prefill kernels), DRAFT_MODE=dflash adaptive depth num_speculative_tokens=7 + adaptive_k (SPEC_N is REFUSED in dflash mode at 1.8.0; another depth goes through EXTRA_ARGS --speculative-config), NCCL_P2P_LEVEL=SYS, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False, VLLM_CUSTOM_ALLREDUCE_ALGO=2stage, VLLM_AMPERE_DECODE_KERNELS=1, sm_80 prefill kernels default on, VLLM_THIN_GEMM=1, VLLM_THIN_GEMM_CONCAT=1, VLLM_HOST_ALLREDUCE=1, VLLM_MOE_DECODE_ROUTE_V2=1, VLLM_MOE_DETERMINISTIC_ALIGN=1, VLLM_MOE_MASK_PADDING=1, VLLM_SHARED_EXPERT_REORDER=1, VLLM_FLA_PIN_AUTOTUNE=1, VLLM_AMPERE_MARLIN_DECODE_CUDA=1, VLLM_AMPERE_MARLIN_DECODE_V2=1 (TP4 only), VLLM_AMPERE_KDA_PREFILL_NARROW=1 + VLLM_AMPERE_KDA_PREFILL_WIDE=1, VLLM_AMPERE_KDA_FOLD=1, VLLM_AMPERE_KDA_SMALL_STACK=1, VLLM_AMPERE_MARLIN_PREFILL_NARROW=1, VLLM_AMPERE_MARLIN_PREFILL_WIDE=1, VLLM_AMPERE_SPARSE_MLA_PREFILL_WIDE=1, VLLM_INDEXER_DECODE_RAW_K=1, VLLM_TOOL_CHOICE_NONE_MASK=1, VLLM_SPARSE_INDEXER_MAX_LOGITS_MB=128 (128 at 1.8.0, was 512 in 1.7.2 docs), VLLM_DFLASH_BOUNDARY_CACHE=1, VLLM_STATE_INDEX_CHECK=1 (engine validation default), VLLM_CUSTOM_ALLREDUCE_PCIE_P2P=1 (default under TP4; under PP4 1.8.0 no longer exports it — TP=1 has no tensor-parallel all-reduce), VLLM_PP_DRAFT_TAIL_STAGE=2, VLLM_PP_SPREAD_DECODES=1, VLLM_PP_PACKED_HOP=1, VLLM_PP_HOP_NO_METADATA=1, VLLM_PP_SPLIT_DRAFT_EVENT=1. container.env: LAYOUT=pp4, P2P=auto, HF_HOME/FLASHINFER_WORKSPACE_BASE/TORCHINDUCTOR_CACHE_DIR under ./cache so the first-boot compile happens once and later boots reuse it. RENAMED at 1.8.0 — the engine logs "... is retired and ignored" once and keeps running at the CODE DEFAULT (many are OFF), so old names in .env/container.env lose features silently instead of erroring; rename them: VLLM_GLM5_DECODE_KERNELS -> VLLM_AMPERE_DECODE_KERNELS, VLLM_GLM5_DECODE_IDX_GLUE -> VLLM_AMPERE_DECODE_IDX_GLUE, VLLM_GLM5_DECODE_MOE_ROUTE_V2 -> VLLM_MOE_DECODE_ROUTE_V2, VLLM_GLM5_DETERMINISTIC_MOE_ALIGN -> VLLM_MOE_DETERMINISTIC_ALIGN, VLLM_GLM5_MOE_MASK_PADDING -> VLLM_MOE_MASK_PADDING, VLLM_GLM5_THIN_GEMM -> VLLM_THIN_GEMM, VLLM_GLM5_HOST_ALLREDUCE -> VLLM_HOST_ALLREDUCE, VLLM_GLM5_SHARED_EXPERT_REORDER -> VLLM_SHARED_EXPERT_REORDER, VLLM_GLM5_FLA_PIN_AUTOTUNE -> VLLM_FLA_PIN_AUTOTUNE, VLLM_GLM5_INDEXER_DECODE_RAW_K -> VLLM_INDEXER_DECODE_RAW_K, VLLM_GLM5_TOOL_CHOICE_NONE_MASK -> VLLM_TOOL_CHOICE_NONE_MASK, VLLM_GLM5_DFLASH_BOUNDARY_CACHE -> VLLM_DFLASH_BOUNDARY_CACHE, VLLM_GLM5_MARLIN_DECODE_CUDA -> VLLM_AMPERE_MARLIN_DECODE_CUDA, VLLM_GLM5_TP4_KDA_PREFILL -> VLLM_AMPERE_KDA_PREFILL_NARROW, VLLM_GLM5_TP4_MARLIN_PREFILL -> VLLM_AMPERE_MARLIN_PREFILL_NARROW, VLLM_GLM5_PP_KDA_PREFILL -> VLLM_AMPERE_KDA_PREFILL_WIDE, VLLM_GLM5_PP_SPARSE_MLA_PREFILL -> VLLM_AMPERE_SPARSE_MLA_PREFILL_WIDE, VLLM_GLM5_PP_MARLIN_PREFILL -> VLLM_AMPERE_MARLIN_PREFILL_WIDE, VLLM_GLM5_PP_MARLIN_PREFILL_COMPILED -> VLLM_AMPERE_MARLIN_PREFILL_WIDE_COMPILED, VLLM_ALLOW_PCIE_P2P_CUSTOM_ALLREDUCE -> VLLM_CUSTOM_ALLREDUCE_PCIE_P2P, VLLM_GLM5_STATE_INDEX_CHECK -> VLLM_STATE_INDEX_CHECK (VLLM_GLM5_TP4_MARLIN_PREFILL survives in 1.8.0 serve.sh only as a leftover comment — not authoritative). NOT READ at 1.8.0 (no warning; no-op at 1.8.0, setting them has no effect): VLLM_GLM5_PROLOGUE_FUSE, VLLM_GLM5_DECODE_KDA_V2, VLLM_GLM5_REPLICATED_EMBED, VLLM_GLM5_SPARSE_MLA_DECODE_LEGACY. VLLM_GLM5_LOCAL_LOGITS=0 became EXTRA_ARGS=--no-enable-batch-sharded-sampling; VLLM_GLM5_DFLASH_ADAPTIVE_K (+_K_DEPTHS/_K_ACCEPT/_K_COSTS/_K_COSTS_MULTI) and VLLM_GLM5_DFLASH_ADAPTIVE_DRAFT_WIDTH collapse into the single --speculative-config adaptive_k above; removed with no replacement (also silently ignored): VLLM_GLM5_PREFILL_OVERLAP (+_BACKEND/_SPLITS), VLLM_GLM5_MARLIN_DECODE_VARIANT, VLLM_GLM5_INDEXER_DECODE_ROWS, VLLM_GLM5_INDEXER_GATHER_CLAMP, VLLM_KV_SWA_INFLIGHT_SCRATCH, VLLM_GLM5_TP4_MARLIN_PREFILL_COMPILED, VLLM_GLM5_THIN_GEMM_V74, VLLM_GLM5_DECODE_MHC_V3, VLLM_GLM5_DECODE_MHC_V2_FN_BF16, VLLM_GLM5_PREFILL_PACK_BF16X2, VLLM_CUSTOM_ALLREDUCE_FLAGS_BUILD_DIR. INSTALL at 1.8.0: container-only — RUNTIME=native, the venv/native build, the install.sh venv path and the native docs are gone; ./start.sh or docker compose up does the whole first start; delete from .env: RUNTIME, VLLM_COMMIT, VLLM_REPO, VLLM_BRANCH, VLLM_PRECOMPILED_WHEEL_COMMIT, VENV, VLLM_SRC, CUDA_HOME, PYTHON_VERSION, BUILD_FROM_SOURCE, MAX_JOBS, NATIVE_INSTALL_RESUME, FLASHINFER_INSTALL_*, TORCH_CUDA_ARCH_LIST, TORCH_INDEX_URL, VLLM_BUILD_AMPERE_MARLIN (the last three named ones are warned about until removed; the image already carries the optional Marlin library).' + notes: 'AS-RUN PP4 box 2026-09-28: util cut 0.95 -> 0.93 (0.95 filled the last card to ~70 MiB free; a 72 MiB KDA prefill workspace OOM-killed warmup). PP4 layout: activations-only inter-card traffic, tuned for x4-link cards; start.sh checks link width and suggests LAYOUT=pp4 automatically. The image runs at native-install speed; 1.8.0 is container-only, so no native install exists. All sm_80 feature flags remain kill switches via env (VLLM_AMPERE_DECODE_KERNELS, FAIR_PREFILL etc.). FIELD REPORT now historical (2026-10-13, site owner): the 1.7.x "off the rails" reports were resolved in 1.7.2 and the 2026-10-08 reports resolved as of 1.7.2; superseded by release 1.8.0 (2026-10-16). KEEPING the 1.6.0 image (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) as the documented known-good fallback: if output coherence degrades on your rig, trying 1.6.0 before debugging hardware remains reasonable.' performance: claimed_by: submitter metrics: @@ -88,25 +90,25 @@ artifacts: kind: drafter pointer_type: hf_model pointer: 'https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2' - pin: 'DFlash2 drafter ~2.2 GiB, k=3 in this recipe' + pin: 'DFlash2 drafter ~2.2 GiB, adaptive depth at 1.8.0 (num_speculative_tokens 7 with adaptive_k by_load [7,5,3]; k=3 was the pre-1.8.0 fixed depth)' - kind: engine_image pointer_type: docker_digest pointer: 'https://github.com/Morrowmake/vllm-cmp170hx' - pin: 'image ghcr.io/morrowmake/vllm-cmp170hx:1.7.2-ab60b723ad, digest sha256:0d249835cb8502316403350a3cad9d531b46e5249e4076ecb70faa42067b5b7a (recipe release 1.7.2). Public image tags: 1.7.2-ab60b723ad, 1.6.0-3a2bf16dae, 1.5.0-e77f89da20. The 1.7.2 serve.sh is launch-compatible with this as-run PP4 config (same dflash k=3 path); the perf numbers below remain release-1.4.1 measurements (1.7.2 validates speeds unchanged within noise). (1.6.0-3a2bf16dae retained as a fallback tag per field reports; 1.7.2 reported working fine 2026-10-13.)' + pin: 'image ghcr.io/morrowmake/vllm-cmp170hx:engine-819146023f, digest sha256:18a2f26207568128e1d792ce923e86a74e40def1e43e492db83132cab8cdbe14 (recipe release 1.8.0). Image tags now name the engine source, independent of recipe release numbers. The 1.8.0 serve.sh launch differs from the as-run 1.7.2 command: DFlash is adaptive depth (num_speculative_tokens 7 + adaptive_k; SPEC_N refused in dflash mode) and --long-prefill-token-threshold 0 is explicit; PP4 batch/block/fair-prefill values are unchanged (2312 / 4608 / 384+2). The perf numbers below are the author 1.8.0 measurements (2026-10-16). (1.6.0-3a2bf16dae retained as the documented known-good fallback per field reports; the 1.7.x field report is historical — 1.8.0 supersedes it.)' - kind: reference pointer_type: git_commit pointer: 'https://github.com/Morrowmake/glm53-flash-cmp170hx-recipe' - pin: 'release 1.7.2 (engine source; image tags on GHCR: 1.7.2-ab60b723ad, 1.6.0-3a2bf16dae, 1.5.0-e77f89da20)' + pin: 'release 1.8.0 (engine 819146023f, tag glm53-recipe-1.8.0, rebased on upstream vLLM 4e0681ed56; image tags on GHCR include engine-819146023f and the older 1.7.2-ab60b723ad / 1.6.0-3a2bf16dae)' provenance: source_urls: - - 'https://github.com/Morrowmake/vllm-cmp170hx/tree/ampere-glm53' + - 'https://github.com/Morrowmake/vllm-cmp170hx/tree/ampere' - 'https://huggingface.co/zai-org/GLM-5.3-Flash' - notes: 'Recipe by Morrowmake (github.com/Morrowmake/glm53-flash-cmp170hx-recipe, MIT licence), release 1.4.1 2026-09-27. Mirrored into localairecipes from the local checkout with every pin transcribed verbatim; engine+image+weights are digest/commit-pinned so this exact stack is reproducible. Updated 2026-10-06 from upstream release 1.4.1 to 1.7.2 (engine ab60b723ada, image 1.7.2-ab60b723ad): TP4 long-gen state recovery (#7), prefill scratch OOM (#9), cached-prefix garbage output (#10), Marlin prefill dependency order (#4), startup retry for slow warmup (#5); as-run launch args unchanged (still the 1.7.2 PP4 defaults); measured tok/s on site unchanged (upstream validates speeds unchanged within noise). FIELD REPORT UPDATE 2026-10-13 (site owner): 1.7.2 (image 1.7.2-ab60b723ad) is now reported working fine on 4x CMP 170HX rigs — upstream issues #7/#8 (TP4 long-gen context loss / endless repetition) and #4 (prefill scratch OOM) are confirmed fixed in 1.7.2. KEEPING the 1.6.0 image (ghcr.io/morrowmake/vllm-cmp170hx:1.6.0-3a2bf16dae) as a known-good fallback: if output coherence degrades on your rig, trying 1.6.0 before debugging hardware remains reasonable. The old 2026-10-08 "off the rails" reports are resolved as of 1.7.2.' + notes: 'Recipe by Morrowmake (github.com/Morrowmake/glm53-flash-cmp170hx-recipe, MIT licence), release 1.4.1 2026-09-27. Mirrored into localairecipes from the local checkout with every pin transcribed verbatim; engine+image+weights are digest/commit-pinned so this exact stack is reproducible. Updated 2026-10-06 from 1.4.1 to 1.7.2; updated 2026-10-28 from 1.7.2 to release 1.8.0 2026-10-16 (engine 819146023facb31c04f1f8512e52464835a10d40, image tag engine-819146023f, digest sha256:18a2f26207568128e1d792ce923e86a74e40def1e43e492db83132cab8cdbe14): engine rebased on upstream vLLM 4e0681ed56 (1,144 commits ahead of the 1.7.2 engine ab60b723ada; a rebase, not a linear descendant — merge-base e55d076f89 is the upstream base 1.7.0-1.7.2 sat on, and the 1.7.2 engine line is not an ancestor of 1.8.0), same features and same context/batch/fair-prefill defaults; 21 engine-switch env names renamed (old names silently ignored at code defaults — see the switch list in config.runtime.env); DFlash default is adaptive depth 7 with adaptive_k by_load [7,5,3]; container-only install (native/venv path removed; first boot compiles kernel caches); --host default is now 127.0.0.1 (the published 0.0.0.0 is the as-run box value only); author re-measured throughput/KV tables for 1.8.0 on 2026-10-16. FIELD REPORT now historical (2026-10-13, site owner): the 1.7.x "off the rails" reports were resolved in 1.7.2; superseded by 1.8.0. KEEPING 1.6.0 (image tag 1.6.0-3a2bf16dae) as the documented known-good fallback.' credit: - name: Morrowmake url: 'https://github.com/Morrowmake/glm53-flash-cmp170hx-recipe' max_context_len: 262144 -kv_pool_tokens: 2320328 +kv_pool_tokens: 2088960