{"slug":"glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn","title":"GLM-5.3-Flash VQ Q4_K 1x DGX Spark 1M ctx (JLFN)","summary":"Recipe by JLFN (dgx-recipe, verified on-box 2026-10-17). Baekpica/GLM-5.3-Flash-VQ-GGUF (72 GiB GVQ1 release directory: routed experts vector-quantized with codebook, 316 nonexpert matrices Q4_K, rest source precision; 56 GGUF parts, embedded MTP predictor) served by the ds4-dfm-rs Rust host (ds4-server, CUDA) on one DGX Spark at the model's native 1048576-token context with two banks. Measured at temperature 0: 24.2-24.3 tok/s one stream, 12.1/12.0 each with two streams (aggregate flat - decode is bytes-per-token against the weight stream). Disclosed quality trade: prose PPL 4.8555 vs 8.0797, code-completion 10.5 vs 18.4%, Korean meeting-summary degeneration under greedy.","license":"MIT","tier":"community","beta":false,"revision":{"no":2,"commit_sha":"f5a53e8b06f37327380e8affb1c9ea298e270456","content_hash":"2d63af307f133a15a2baacaee8d45c16f33ff129","created_at":"2026-10-20 21:52:36.476254-06","notes":"Recipe, runbook, launcher and measurements: JLFN/dgx-recipe @ 50f6df432b937b04 (github.com/JLFN/dgx-recipe; GLM VQ lane verified on the reference Spark 2026-10-17 per repo README table). Weights: Baekpica/GLM-5.3-Flash-VQ-GGUF @ 4da527cd0f9e83ccd9fb451a1feaf3ba861f7c77 (GVQ1 Q4_K nonexpert v3; 56 parts + tables verified via HF API this ingest; SHA256SUMS verified by the engine at load per runbook). Engine: Baekpica/ds4-dfm-rs (Rust host of antirez/ds4), main from c5384b5e PR86; measured on optimize4 checkout. Not verified upstream: long-context behaviour at 1M, predictor benefit, vision tower (launcher serves text only)."},"engine_source":{"type":"fork","git_url":"https://github.com/Baekpica/ds4-dfm-rs","base_engine":"other","pip_package":null,"base_version":"ds4-server (ds4-dfm-rs main @ c5384b5e+)","image_digest":null,"install_note":"Not pip/docker-installable: build ds4-dfm-rs from source on the Spark (make cuda-spark; CUDA 13 runtime; engine main from c5384b5e PR86). Weights are a GVQ1 release DIRECTORY (56 GGUF parts + files.json/tensor-manifest/tokenizer/chat_template beside them) - flattening breaks it.","pinned_commit":"c5384b5e"},"credit":[{"name":"JLFN","url":"https://github.com/JLFN/dgx-recipe"},{"name":"Baekpica","url":"https://huggingface.co/Baekpica"}],"author":{"name":"localai-recipes","isSite":true},"uploaded_by":"localai-recipes","config":{"launch":["bash scripts/start-glm53-flash-vq.sh preflight","bash scripts/start-glm53-flash-vq.sh download","bash scripts/start-glm53-flash-vq.sh check","DS4_CONT_MTP_MODE=2 DS4_CONT_MTP_DEPTH=2 ./ds4-server --cuda","-m ~/models/GLM-5.3-Flash-VQ-GGUF","--mtp-mode on --mtp-draft 2","--host 0.0.0.0 --port 8007","--model-id glm-5.3-flash-vq --no-update-check"],"max_context_len":1048576,"storage_requirements":{"note":"72 GiB GVQ1 release directory on the roomy volume; read once at load (9.9 s device-cache stage), then mmap-resident.","ram_gb":121.599999999999994315658113919198513031005859375,"nvme_gb":87}},"models":[{"slug":"glm-5-3-flash-vq-q4k","name":"GLM-5.3-Flash-VQ-Q4K","hf_repo":"Baekpica/GLM-5.3-Flash-VQ-GGUF","quant":"q4-k-vq-nonexpert-v3"}],"hardware":[{"slug":"dgx-spark","vendor":"nvidia","card":"DGX Spark (GB10, unified memory)","card_family":"dgx-spark","vram_gb":128,"device_count":1}],"generation":"blackwell","tags":["1m-ctx","dgx-spark","gb10","long-context","mtp","q4k","single-gpu","sm121","vector-quantized"],"tasks":["chat","code"],"aa":{"aa_index":42,"aa_provenance":"measured","aa_source_url":"https://artificialanalysis.ai/models/glm-5-3-flash","aa_index_version":null,"aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-04 14:15:00-06","aa_family_slug":"glm-5-3-flash","aa_resolved_slug":"glm-5-3-flash"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Reference Spark 2026-10-17, temperature 0, 256-token completions, native context, two banks. context_len marker: 1048576 = c1 decode; 1069056 (+480) = TTFT no contention; 1089536 (+960) = TTFT with 2-prompt contention; 1106016 (+7440) = c2 per-stream. Aggregate flat (pair wall 22.67 s = sequential 22.65 s): decode is bytes-per-token against the weight stream. tok_per_step 1.0 in every run: MTP loaded+offered, not committed on this build (DS4_CONT_MTP_MODE=2/DEPTH=2 confirmed in-process). Artifact card 34.6-40.6 tok/s = bench path, not this HTTP lane.\", \"tok_s\": 24.300000000000000710542735760100185871124267578125, \"points\": [{\"notes\": \"Single-stream decode, two runs: 24.2 and 24.3 tok/s (temp 0, 256-token completions). context_len is the tuple marker: model-native 1048576 + 0.\", \"concurrency\": 1, \"context_len\": 1048576, \"decode_tok_s\": 24.300000000000000710542735760100185871124267578125}, {\"notes\": \"TTFT 222 ms single stream (no contention). context_len is the tuple marker: 1048576 + 480.\", \"ttft_ms\": 222, \"context_len\": 1048576}, {\"notes\": \"Two concurrent streams: 12.1 and 12.0 tok/s each; pair wall 22.67 s vs 22.65 s sequential. context_len is the tuple marker: 1048576 + 7440.\", \"concurrency\": 2, \"context_len\": 1106016, \"decode_tok_s\": 12.0999999999999996447286321199499070644378662109375}, {\"notes\": \"TTFT 1592 ms when two prompts arrive together (222 ms alone). context_len is the tuple marker: 1048576 + 960.\", \"ttft_ms\": 1592, \"context_len\": 1089536}], \"ttft_ms\": 222, \"context_len\": 1048576}"}],"perf_points":[{"id":1627,"context_len":1048576,"prefill_tok_s":null,"decode_tok_s":null,"ttft_ms":222,"concurrency":null,"claimed_by":"submitter","notes":"TTFT 222 ms single stream (no contention). context_len is the tuple marker: 1048576 + 480."},{"id":1626,"context_len":1048576,"prefill_tok_s":null,"decode_tok_s":24.300000000000000710542735760100185871124267578125,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"Single-stream decode, two runs: 24.2 and 24.3 tok/s (temp 0, 256-token completions). context_len is the tuple marker: model-native 1048576 + 0."},{"id":1628,"context_len":1089536,"prefill_tok_s":null,"decode_tok_s":null,"ttft_ms":1592,"concurrency":null,"claimed_by":"submitter","notes":"TTFT 1592 ms when two prompts arrive together (222 ms alone). context_len is the tuple marker: 1048576 + 960."},{"id":1629,"context_len":1106016,"prefill_tok_s":null,"decode_tok_s":12.0999999999999996447286321199499070644378662109375,"ttft_ms":null,"concurrency":2,"claimed_by":"submitter","notes":"Two concurrent streams: 12.1 and 12.0 tok/s each; pair wall 22.67 s vs 22.65 s sequential. context_len is the tuple marker: 1048576 + 7440."}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn.json"},"requirements":null,"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/Baekpica/GLM-5.3-Flash-VQ-GGUF","pin":"4da527cd0f9e83ccd9fb451a1feaf3ba861f7c77"},{"kind":"engine_image","pointer_type":"git_commit","pointer":"git+https://github.com/Baekpica/ds4-dfm-rs","pin":"c5384b5e"},{"kind":"reference","pointer_type":"git_commit","pointer":"git+https://github.com/JLFN/dgx-recipe","pin":"50f6df432b937b04d2f47fd4332d258b838b2dd0"}],"links":{"self":"/api/v1/recipes/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn.json","alternate":"/r/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn","describedby":"/openapi.json"},"updated_at":"2026-10-20 21:52:36.476254-06","created_at":"2026-10-20 21:49:15.295039-06","speeds":{"decode_tok_s":24.300000000000000710542735760100185871124267578125,"prefill_tok_s":0,"leaderboard":"/api/v1/leaderboard.json"}}