{"slug":"glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4","title":"GLM-5.3-Flash exlr8 EXL3 experts on DGX Spark TP1-4","summary":"GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3/4 bpw), chosen at boot via VLLM_EXLR8_WIDTHS. Reference topology TP=4 on four GB10 boxes; TP=1 serves the whole model (~87 GiB) on one 128 GB DGX Spark / ASUS GX10 via tp1.yaml plus a distinct MENTAT_GROUP. TP=2 and TP=3 also measured. 2026-10-19, clocks TP-locked at 2200 MHz: decode code 128.0 tok/s, prose 72.5 at TP=4; 53.0 / 29.5 at TP=1; prefill 4,393 tok/s at 8K, 4,190 at 32K. Trade vs the shipping NVFP4 build: decode +3-8%, NLL equal or better, KV pool ~+15%, prefill 7.7% lower at Second 1x-Spark GLM-5.3-Flash lane: not the VQ Q4_K GGUF lane (/r/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn), and not a fork of the 2x ATLAS build (/r/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m) whose NVFP4 Flash base this stack overlays.","license":"MIT","tier":"community","beta":false,"revision":{"no":4,"commit_sha":"2ddd4d836d8efa003f7203ba30597c20dc21e495","content_hash":"79db03010b10026e9788b7c749df25daf5fb38be","created_at":"2026-10-29 21:32:25.385656-06","notes":"Upstream pinned at main @ 6f5ede7f2d2675fadbb9bb216a34b28386547c27 (re-read live 2026-10-23). Figures from README.md (author campaign 2026-10-19, clocks TP-locked at 2200 MHz); stack and knob detail from NOTES.md, KNOBS.md, experimental/compose/exlr8.yaml and tp1.yaml at that sha. corrections_to_card: (1) card called this the first single-DGX-Spark GLM-5.3-Flash recipe; that is wrong -- glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn already carries a 1x Spark lane, so this is a SECOND single-box lane on a different weight path (EXL3-trellis experts vs the VQ Q4_K GGUF lane); cross-linked to that recipe and to glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m. (2) the card's perf table put 2,609 in the TP=2 decode-prose column; README gives 2,609 as TP=2 cold prefill at 8K and 50.1 tok/s for TP=2 decode prose. (3) card asserted MIT for the recipe repo; GitHub reports no license for it and there is no LICENSE file at the pinned sha -- MIT is asserted here for the redistributed expert units and the base NVFP4 weights, not the recipe repo. (4) no upstream release tag exists; the reference is the pinned commit."},"engine_source":{"type":"fork","git_url":"https://github.com/kindlingai/glm-5.3-flash-exlr8-gx10","base_engine":"vllm","pip_package":null,"base_version":"vllm-exlr8@a2e163cb05","image_digest":null,"install_note":"Source build: docker build -f image/Dockerfile . The overlay sets --quantization exlr8 --load-format exlr8 over the NVFP4 Flash base image. The vllm engine tree arrives via the repo symlink vllm -> .submodules/spark-agent/vllm (submodule mmastrac/spark-agent); the fork revision a2e163cb05 is no longer resolvable in any public repo.","pinned_commit":"a2e163cb05"},"credit":[{"name":"kindlingai","url":"https://github.com/kindlingai/glm-5.3-flash-exlr8-gx10"},{"name":"nvidia","url":"https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4"}],"author":{"name":"localai-recipes","isSite":false},"uploaded_by":"localai-recipes","config":{"launch":["sudo env MODELS_ROOT=/mnt/nfs/models ./glm53f up -d","sudo env TP=2 VLLM_EXLR8_WIDTHS=k3 ./glm53f up -d","sudo env TP=1 VLLM_EXLR8_WIDTHS=k2 MENTAT_GROUP=glm53-solo ./glm53f -f experimental/compose/tp1.yaml up -d","sudo env TP=3 VLLM_EXLR8_WIDTHS=k4 MODEL_DIR=/models/nas/glm-5.3-flash/exlr8-release-tp3 ./glm53f -f experimental/compose/tp3.yaml up -d"],"layouts":["tp"],"kv_pool_tokens":6459582,"max_context_len":524288,"storage_requirements":{"note":"Weights 181.3 GiB for the NVFP4 checkpoint (NOTES.md:73, safetensors indexes). TP=1 holds the full expert set; plan ~182 GiB local disk.","ram_gb":128,"nvme_gb":182}},"models":[{"slug":"glm-5-3-flash-exl3","name":"GLM-5.3-Flash-EXL3","hf_repo":"kindlingai/glm-5.3-flash-exlr8-k2-k3-k4","quant":"exlr8-k2-k3-k4"}],"hardware":[{"slug":"dgx-spark-x3","vendor":"nvidia","card":"DGX Spark (GB10, unified memory)","card_family":"dgx-spark","vram_gb":128,"device_count":3},{"slug":"dgx-spark","vendor":"nvidia","card":"DGX Spark (GB10, unified memory)","card_family":"dgx-spark","vram_gb":128,"device_count":1},{"slug":"dgx-spark-x2","vendor":"nvidia","card":"DGX Spark (GB10, unified memory)","card_family":"dgx-spark","vram_gb":128,"device_count":2},{"slug":"dgx-spark-x4","vendor":"nvidia","card":"DGX Spark (GB10, unified memory)","card_family":"dgx-spark","vram_gb":128,"device_count":4}],"generation":"blackwell","tags":[],"tasks":["chat","code","vision"],"aa":{"aa_index":42,"aa_provenance":"measured","aa_source_url":"https://artificialanalysis.ai/models/glm-5-3-flash","aa_index_version":null,"aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-04 14:15:00-06","aa_family_slug":"glm-5-3-flash","aa_resolved_slug":"glm-5-3-flash"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Headline 53.0 tok/s = TP=1 single-box k2 code decode, the topology that makes this a distinct single-box lane. Transcribed matrix (README.md, author campaign 2026-10-19, clocks TP-locked 2200 MHz), decode code / prose: TP=1 53.0 / 29.5, TP=2 84.2 / 50.1, TP=3 105.8 / 57.8, TP=4 128.0 / 72.5. Cold prefill 8K / 32K: TP=1 495 / 523, TP=2 2,609 / 2,687, TP=3 3,002 / 3,391, TP=4 4,393 / 4,190. Figures are the repo's own receipts, not re-measured here.\", \"tok_s\": 53, \"points\": [{\"notes\": \"TP=4 k4 code decode\", \"concurrency\": 1, \"context_len\": 524288, \"decode_tok_s\": 128}, {\"notes\": \"TP=4 k4 prose decode; context_len marker 524288+8192\", \"concurrency\": 1, \"context_len\": 532480, \"decode_tok_s\": 72.5}, {\"notes\": \"TP=3 k4 code decode; context_len marker 524288+16384\", \"concurrency\": 1, \"context_len\": 540672, \"decode_tok_s\": 105.7999999999999971578290569595992565155029296875}, {\"notes\": \"TP=3 k4 prose decode; context_len marker 524288+24576\", \"concurrency\": 1, \"context_len\": 548864, \"decode_tok_s\": 57.7999999999999971578290569595992565155029296875}, {\"notes\": \"TP=2 k3 code decode; context_len marker 524288+32768\", \"concurrency\": 1, \"context_len\": 557056, \"decode_tok_s\": 84.2000000000000028421709430404007434844970703125}, {\"notes\": \"TP=2 k3 prose decode; context_len marker 524288+40960\", \"concurrency\": 1, \"context_len\": 565248, \"decode_tok_s\": 50.10000000000000142108547152020037174224853515625}, {\"notes\": \"TP=1 k2 code decode; context_len marker 524288+49152\", \"concurrency\": 1, \"context_len\": 573440, \"decode_tok_s\": 53}, {\"notes\": \"TP=1 k2 prose decode; context_len marker 524288+57344\", \"concurrency\": 1, \"context_len\": 581632, \"decode_tok_s\": 29.5}, {\"notes\": \"TP=4 k4 cold prefill 8K, headline topology\", \"context_len\": 8192, \"prefill_tok_s\": 4393}, {\"notes\": \"TP=4 k4 cold prefill 32K, headline topology\", \"context_len\": 32768, \"prefill_tok_s\": 4190}, {\"notes\": \"TP=2 k3 cold prefill 8K\", \"context_len\": 8192, \"prefill_tok_s\": 2609}, {\"notes\": \"TP=2 k3 cold prefill 32K\", \"context_len\": 32768, \"prefill_tok_s\": 2687}], \"context_len\": 524288}"}],"perf_points":[{"id":2054,"context_len":8192,"prefill_tok_s":4393,"decode_tok_s":null,"ttft_ms":null,"concurrency":null,"claimed_by":"submitter","notes":"TP=4 k4 cold prefill 8K, headline topology"},{"id":2055,"context_len":8192,"prefill_tok_s":2609,"decode_tok_s":null,"ttft_ms":null,"concurrency":null,"claimed_by":"submitter","notes":"TP=2 k3 cold prefill 8K"},{"id":2056,"context_len":32768,"prefill_tok_s":4190,"decode_tok_s":null,"ttft_ms":null,"concurrency":null,"claimed_by":"submitter","notes":"TP=4 k4 cold prefill 32K, headline topology"},{"id":2057,"context_len":32768,"prefill_tok_s":2687,"decode_tok_s":null,"ttft_ms":null,"concurrency":null,"claimed_by":"submitter","notes":"TP=2 k3 cold prefill 32K"},{"id":2058,"context_len":524288,"prefill_tok_s":null,"decode_tok_s":128,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=4 k4 code decode"},{"id":2059,"context_len":532480,"prefill_tok_s":null,"decode_tok_s":72.5,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=4 k4 prose decode; context_len marker 524288+8192"},{"id":2060,"context_len":540672,"prefill_tok_s":null,"decode_tok_s":105.7999999999999971578290569595992565155029296875,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=3 k4 code decode; context_len marker 524288+16384"},{"id":2061,"context_len":548864,"prefill_tok_s":null,"decode_tok_s":57.7999999999999971578290569595992565155029296875,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=3 k4 prose decode; context_len marker 524288+24576"},{"id":2062,"context_len":557056,"prefill_tok_s":null,"decode_tok_s":84.2000000000000028421709430404007434844970703125,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=2 k3 code decode; context_len marker 524288+32768"},{"id":2063,"context_len":565248,"prefill_tok_s":null,"decode_tok_s":50.10000000000000142108547152020037174224853515625,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=2 k3 prose decode; context_len marker 524288+40960"},{"id":2064,"context_len":573440,"prefill_tok_s":null,"decode_tok_s":53,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=1 k2 code decode; context_len marker 524288+49152"},{"id":2065,"context_len":581632,"prefill_tok_s":null,"decode_tok_s":29.5,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"TP=1 k2 prose decode; context_len marker 524288+57344"}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4.json"},"requirements":null,"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/kindlingai/glm-5.3-flash-exlr8-k2-k3-k4","pin":"85813091679dd187cb7f72fd455ad12fdff20f77"},{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4","pin":"nvidia/GLM-5.3-Flash-NVFP4"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/kindlingai/glm-5.3-flash-exlr8-gx10","pin":"6f5ede7f2d2675fadbb9bb216a34b28386547c27"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/kindlingai/glm-5.3-flash-gx10","pin":"mmastrac/glm-5.3-flash-4x-gx10@0d1f462"}],"links":{"self":"/api/v1/recipes/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4.json","alternate":"/r/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4","describedby":"/openapi.json"},"updated_at":"2026-10-29 21:32:25.385656-06","created_at":"2026-10-24 10:35:19.385671-06","speeds":{"decode_tok_s":65.2000000000000028421709430404007434844970703125,"prefill_tok_s":3438.5,"leaderboard":"/api/v1/leaderboard.json"}}