{"slug":"glm-5-3-flash-max-effort-on-2x-cmp170hx-exl3-tp2-four-slots","title":"GLM-5.3-Flash max effort on 2x CMP170HX EXL3 TP2 four slots","summary":"Four native active sequences on two 64 GiB CMP170HX cards; GLM-5.3-Flash EXL3 3.05bpw, TP2, with the Weightless GLP44 steering vector at alpha 2, FP16 recurrent state, Q8 GPU KV, DFlash2 K7 and hybrid15. This recipe records max reasoning effort, selected per request with model GLM-5.3-Flash-max. llm-inference-bench v0.8.0, unmodified under a per-agent counting wrapper, measures 1, 2 and 4 agents on short and nominal 32K/128K inputs, one 30-second run per cell; actual tokenized input lengths are recorded. Native completion-token usage includes reasoning. Active RAM KV paging is disabled. The 655,360-token pool is shared, with 392,960 including output as the per-request cap. Throughput is inference speed, not an accuracy or intelligence score. Differences between the low, high and max recipes are not separated from run-to-run variation, and with ignore-EOS effort changes what the model writes, not how many tokens. Accepts optional chat_template_kwargs.clear_thinking (default-off).","license":"CC-BY-4.0","tier":"community","beta":false,"revision":{"no":5,"commit_sha":"663174ab27ccbc0f93296953b897b19e08b64726","content_hash":"1bf0157fd231d61ac68175f7a1493df8e9583746","created_at":"2026-10-29 23:29:37.835428-06","notes":"Four-slot configuration of the fork behind the site-owned two-slot listing; this prose workload is not comparable with that listing's numbers. Exact measured source profile is packaged under profiles/community-four-active. Engine stays GPU-resident; Weightless steering vector enabled. Effort is selected per API request via the documented model alias. Inference speed does not establish accuracy or intelligence; no local AA score is asserted. PROD SYNC (kanban t_3ad93f24, owner go Casey 2026-10-28): replay of QC-approved preprod revision r3 content_hash 0b2f1599d710d60751862fce4aa071073828d53a (parent QC t_4c7cb164 APPROVE); prod pre-state content_hash a05d0aa17b178237398ef71a96d99829c5659f82 captured for rollback; perf points carried unchanged (prod lineage)."},"engine_source":{"type":"fork","git_url":"https://github.com/alexvnesta/glm53-flash-cmp170hx-tp","base_engine":"exllamav3","pip_package":null,"base_version":"0f685aa + row32 four-slot profile","image_digest":null,"install_note":"Source build required; no image. Follow https://github.com/alexvnesta/glm53-flash-cmp170hx-tp/blob/db01c256a8003daf90c932ee25552526ede40f1f/profiles/community-four-active/README.md. Set up the pinned venv; apply_engine.sh; install gguf==0.17.1; profiles/community-four-active/build.sh /ABS/profile; get_models.sh using the prepared engine/native directories; obtain the gated GLP44 vector under your own account; launch run.sh with GLM53_COMMUNITY_PROFILE and GLM_WEIGHTLESS_VECTOR. Rebuilt extension hashes vary with build paths. Two 64 GiB cards, CUDA 13.3, Python 3.13 and the pinned Torch environment were used.","pinned_commit":"db01c256a8003daf90c932ee25552526ede40f1f"},"credit":[{"name":"alexvnesta (tensor-parallel fork: TP2, hybrid drafting, concurrent sessions, RAM tier, vision, NCCL backend)","url":"https://github.com/alexvnesta/glm53-flash-cmp170hx-tp"},{"name":"Flun (original 2-card layer-split server + recipe this fork starts from)","url":"https://github.com/Flun/glm53-flash-cmp170hx-exl3"},{"name":"turboderp (ExLlamaV3 engine + GLM-5.3-Flash-exl3 checkpoint + GLM-5.3 TP + vision tower support in engine)","url":"https://github.com/turboderp-org/exllamav3"},{"name":"incoai (GLM-5.3-Flash-DFlash2 BF16 drafter, CC BY-NC-ND 4.0)","url":"https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2"},{"name":"0xSero (k_hcfuse fused hyper-connection kernels, via Flun's repo)","url":"https://github.com/sybil-solutions/glm-flash-lite"},{"name":"Morrowmake (content-verified P2P check idea + benchmark prompt style)","url":"https://github.com/Morrowmake/glm53-flash-cmp170hx-recipe"},{"name":"bayley + arvkmr (fault probe idea + router-precision findings, vllm-170hx-glm5)","url":"https://github.com/bayley/vllm-170hx-glm5"},{"name":"Z.ai (GLM-5.3-Flash base model)","url":"https://huggingface.co/zai-org/GLM-5.3-Flash"},{"name":"msuiche (Weightless GLP-44 vector and specification)","url":"https://huggingface.co/msuiche/GLM-5.3-Flash-abliterated-cyber-GLP-44"}],"author":{"name":"alex.v.nesta","isSite":false},"uploaded_by":"alex.v.nesta","config":{"launch":["./profiles/community-four-active/run.sh"],"layouts":["tp"],"runtime":{"notes":"RECURRENT_STATE_DTYPE sets the recurrent-state storage dtype: fp32 default (launch adds --recurrent_state_fp32); fp16 opts into the upstream FP16 recurrent storage patch (0005-upstream-fp16-recurrent-state); accepts only fp16|fp32, anything else exits 2 at launch."},"active_kv":"GPU-resident Q8 pool 655360 shared tokens, 392960 cap including output per request; no active RAM paging","report_url":"https://github.com/alexvnesta/glm53-flash-cmp170hx-tp/blob/db01c256a8003daf90c932ee25552526ede40f1f/docs/COMMUNITY_BENCHMARK_2026-10-17.md","weightless":{"alpha":2,"layers":"1-44","vector_sha256":"0ccce6b748f87da81505ff2bc3ca82429110940b6c0def60063d304785ccc00c","vector_revision":"ef85b0169fd25a3c0223eedddee124cf6b1580dc"},"environment":{"GLM_WEIGHTLESS_VECTOR":"/absolute/path/to/GLM-5.3-Flash-abliterated-cyber-GLP-44-L1-44-a2.gguf","RECURRENT_STATE_DTYPE":"fp32","GLM53_COMMUNITY_PROFILE":"/absolute/path/to/community-profile","GLM_PERSISTENT_CACHE_DIR":"/absolute/path/to/session-cache","GLM_PERSISTENT_CACHE_GIB":"50","GLM_PERSISTENT_CACHE_MIN_FREE_GIB":"16"},"client_model":"GLM-5.3-Flash-max","native_layout":"4 active sequences, row32 verifier, K7 DFlash2 plus hybrid15, native broadcast/gather with NCCL reductions","kv_pool_tokens":655360,"request_defaults":{"reasoning_effort":"max"},"storage_requirements":{"note":"Tested host RAM, not an established minimum. Inactive KV: 8 GiB RAM; recurrent/checkpoints: 5 GiB; optional disk cache: 50 GiB.","ram_gb":128}},"models":[{"slug":"glm-5-3-flash-exl3-3-05bpw","name":"GLM-5.3-Flash-EXL3-3.05bpw","hf_repo":"turboderp/GLM-5.3-Flash-exl3","quant":"exl3-3-05bpw"}],"hardware":[{"slug":"cmp-170hx-x2","vendor":"nvidia","card":"cmp-170hx","card_family":"170hx","vram_gb":64,"device_count":2}],"generation":"ampere","tags":["concurrent-sessions","dflash2","exl3","q8-kv","tensor-parallel"],"tasks":["chat","code","vision"],"aa":{"aa_index":42,"aa_provenance":"measured","aa_source_url":"https://artificialanalysis.ai/models/glm-5-3-flash","aa_index_version":null,"aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-04 14:15:00-06","aa_family_slug":"glm-5-3-flash","aa_resolved_slug":"glm-5-3-flash"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Headline tok_s: one agent, short prompt (78 input tokens). Community prose workload, max effort, temperature 0, 30 s steady-state windows, one run per cell, output cap 2048, immediate backfill and ignore-EOS. All rates are whole-rig. Points use actual input tokens; the run files keep the nominal targets. Prefill not measured. Not comparable with structured-task numbers for the same fork, including the two-slot listing.\", \"tok_s\": 41.4733142771386127378718811087310314178466796875, \"points\": [{\"notes\": \"max effort; nominal context 0; exact native completion counts, 30 s window; actual input 78.\", \"concurrency\": 1, \"context_len\": 78, \"decode_tok_s\": 41.4733142771386127378718811087310314178466796875}, {\"notes\": \"max effort; nominal context 0; exact native completion counts, 30 s window; actual input 78.\", \"concurrency\": 2, \"context_len\": 78, \"decode_tok_s\": 65.3581817056967366852404666133224964141845703125}, {\"notes\": \"max effort; nominal context 0; exact native completion counts, 30 s window; actual input 78.\", \"concurrency\": 4, \"context_len\": 78, \"decode_tok_s\": 101.219270695103062962516560219228267669677734375}, {\"notes\": \"max effort; nominal context 32768; exact native completion counts, 30 s window; actual input 32319.\", \"concurrency\": 1, \"context_len\": 32319, \"decode_tok_s\": 46.91853713865066310972906649112701416015625}, {\"notes\": \"max effort; nominal context 32768; exact native completion counts, 30 s window; actual input 32319.\", \"concurrency\": 2, \"context_len\": 32319, \"decode_tok_s\": 46.3532810475466021671309135854244232177734375}, {\"notes\": \"max effort; nominal context 32768; exact native completion counts, 30 s window; actual input 32319.\", \"concurrency\": 4, \"context_len\": 32319, \"decode_tok_s\": 64.14413122957694213255308568477630615234375}, {\"notes\": \"max effort; nominal context 131072; exact native completion counts, 30 s window; actual input 128878.\", \"concurrency\": 1, \"context_len\": 128878, \"decode_tok_s\": 49.82240937091719246154752909205853939056396484375}, {\"notes\": \"max effort; nominal context 131072; exact native completion counts, 30 s window; actual input 128878.\", \"concurrency\": 2, \"context_len\": 128878, \"decode_tok_s\": 51.77047493200232253229842172004282474517822265625}, {\"notes\": \"max effort; nominal context 131072; exact native completion counts, 30 s window; actual input 128878.\", \"concurrency\": 4, \"context_len\": 128878, \"decode_tok_s\": 67.970195838417197364833555184304714202880859375}], \"context_len\": 78}"}],"perf_points":[{"id":2205,"context_len":78,"prefill_tok_s":null,"decode_tok_s":41.47331427713900353637654916383326053619384765625,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"max effort; nominal context 0; exact native completion counts, 30 s window; actual input 78."},{"id":2214,"context_len":78,"prefill_tok_s":null,"decode_tok_s":41.47331427713900353637654916383326053619384765625,"ttft_ms":331.80228900164001970551908016204833984375,"concurrency":1,"claimed_by":"benchmark","notes":"llm-inference-bench"},{"id":2206,"context_len":78,"prefill_tok_s":null,"decode_tok_s":65.35818170569700669148005545139312744140625,"ttft_ms":null,"concurrency":2,"claimed_by":"submitter","notes":"max effort; nominal context 0; exact native completion counts, 30 s window; actual input 78."},{"id":2207,"context_len":78,"prefill_tok_s":null,"decode_tok_s":101.219270695099993417898076586425304412841796875,"ttft_ms":null,"concurrency":4,"claimed_by":"submitter","notes":"max effort; nominal context 0; exact native completion counts, 30 s window; actual input 78."},{"id":2208,"context_len":32319,"prefill_tok_s":null,"decode_tok_s":46.91853713865099706481487373821437358856201171875,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"max effort; nominal context 32768; exact native completion counts, 30 s window; actual input 32319."},{"id":2209,"context_len":32319,"prefill_tok_s":null,"decode_tok_s":46.35328104754700007106293924152851104736328125,"ttft_ms":null,"concurrency":2,"claimed_by":"submitter","notes":"max effort; nominal context 32768; exact native completion counts, 30 s window; actual input 32319."},{"id":2210,"context_len":32319,"prefill_tok_s":null,"decode_tok_s":64.14413122957699897597194649279117584228515625,"ttft_ms":null,"concurrency":4,"claimed_by":"submitter","notes":"max effort; nominal context 32768; exact native completion counts, 30 s window; actual input 32319."},{"id":2211,"context_len":128878,"prefill_tok_s":null,"decode_tok_s":49.82240937091700061500887386500835418701171875,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"max effort; nominal context 131072; exact native completion counts, 30 s window; actual input 128878."},{"id":2212,"context_len":128878,"prefill_tok_s":null,"decode_tok_s":51.7704749320020027880673296749591827392578125,"ttft_ms":null,"concurrency":2,"claimed_by":"submitter","notes":"max effort; nominal context 131072; exact native completion counts, 30 s window; actual input 128878."},{"id":2213,"context_len":128878,"prefill_tok_s":null,"decode_tok_s":67.9701958384169984128675423562526702880859375,"ttft_ms":null,"concurrency":4,"claimed_by":"submitter","notes":"max effort; nominal context 131072; exact native completion counts, 30 s window; actual input 128878."}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/glm-5-3-flash-max-effort-on-2x-cmp170hx-exl3-tp2-four-slots.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/glm-5-3-flash-max-effort-on-2x-cmp170hx-exl3-tp2-four-slots.json"},"requirements":null,"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/turboderp/GLM-5.3-Flash-exl3","pin":"332ab457b709b7ba30dd9a448be5de03b80a7ac9 (3.05bpw)"},{"kind":"drafter","pointer_type":"hf_model","pointer":"https://huggingface.co/incoai/GLM-5.3-Flash-DFlash2","pin":"bf582e4eacc1810f76656d1811693ff6c6737d2a; convert locally to EXL3 6bpw"},{"kind":"abliteration","pointer_type":"hf_model","pointer":"https://huggingface.co/msuiche/GLM-5.3-Flash-abliterated-cyber-GLP-44","pin":"ef85b0169fd25a3c0223eedddee124cf6b1580dc; L1-44 alpha2 GGUF sha256 0ccce6b748f87da81505ff2bc3ca82429110940b6c0def60063d304785ccc00c"}],"links":{"self":"/api/v1/recipes/glm-5-3-flash-max-effort-on-2x-cmp170hx-exl3-tp2-four-slots.json","alternate":"/r/glm-5-3-flash-max-effort-on-2x-cmp170hx-exl3-tp2-four-slots","describedby":"/openapi.json"},"updated_at":"2026-10-29 23:29:37.835428-06","created_at":"2026-10-17 08:44:55.347529-06","speeds":{"decode_tok_s":41.5,"prefill_tok_s":0,"leaderboard":"/api/v1/leaderboard.json"}}