localairecipes launch configs for local inference

GLM-5.3-Flash VQ Q4_K 1x DGX Spark 1M ctx (JLFN)

0 sign in to vote

community Community submission — verify on your own hardware.

recipe by JLFN↗, Baekpica↗ · listed by localai-recipes

revision r2 · commit f5a53e8b06 · updated 10 d ago · license MIT

chat code 1m-ctx single-gpu long-context dgx-spark gb10 sm121 mtp q4k vector-quantized

Recipe by JLFN (dgx-recipe, verified on-box 2026-10-17). Baekpica/GLM-5.3-Flash-VQ-GGUF (72 GiB GVQ1 release directory: routed experts vector-quantized with codebook, 316 nonexpert matrices Q4_K, rest source precision; 56 GGUF parts, embedded MTP predictor) served by the ds4-dfm-rs Rust host (ds4-server, CUDA) on one DGX Spark at the model's native 1048576-token context with two banks. Measured at temperature 0: 24.2-24.3 tok/s one stream, 12.1/12.0 each with two streams (aggregate flat - decode is bytes-per-token against the weight stream). Disclosed quality trade: prose PPL 4.8555 vs 8.0797, code-completion 10.5 vs 18.4%, Korean meeting-summary degeneration under greedy.

Launch command

'bash scripts/start-glm53-flash-vq.sh preflight' 'bash scripts/start-glm53-flash-vq.sh download' 'bash scripts/start-glm53-flash-vq.sh check' 'DS4_CONT_MTP_MODE=2 DS4_CONT_MTP_DEPTH=2 ./ds4-server --cuda' '-m ~/models/GLM-5.3-Flash-VQ-GGUF' '--mtp-mode on --mtp-draft 2' '--host 0.0.0.0 --port 8007' '--model-id glm-5.3-flash-vq --no-update-check'

Use this recipe with your agent

Paste this into any agent chat — it fetches and applies the recipe:

Fetch https://localairecipes.com/api/v1/recipes/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

Or fetch it directly:

curl -fsSL https://localairecipes.com/api/v1/recipes/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn.json

Engine

Install note: Not pip/docker-installable: build ds4-dfm-rs from source on the Spark (make cuda-spark; CUDA 13 runtime; engine main from c5384b5e PR86). Weights are a GVQ1 release DIRECTORY (56 GGUF parts + files.json/tensor-manifest/tokenizer/chat_template beside them) - flattening breaks it.

Model

Hardware

Context

  • max context: 1M tokens

Performance

Speed by context

ContextConc.PrefillDecodeTTFT
1,048,576———222 msreported
1,048,5761—24.3 tok/s—reported
1,089,536———1592 msreported
1,106,0162—12.1 tok/s—reported

prefill = prompt processing · decode = generation. All decode/prefill figures are shown as reported.

reported

notesReference Spark 2026-10-17, temperature 0, 256-token completions, native context, two banks. context_len marker: 1048576 = c1 decode; 1069056 (+480) = TTFT no contention; 1089536 (+960) = TTFT with 2-prompt contention; 1106016 (+7440) = c2 per-stream. Aggregate flat (pair wall 22.67 s = sequential 22.65 s): decode is bytes-per-token against the weight stream. tok_per_step 1.0 in every run: MTP loaded+offered, not committed on this build (DS4_CONT_MTP_MODE=2/DEPTH=2 confirmed in-process). Artifact card 34.6-40.6 tok/s = bench path, not this HTTP lane.

Downloads

Revision notes

Recipe, runbook, launcher and measurements: JLFN/dgx-recipe @ 50f6df432b937b04 (github.com/JLFN/dgx-recipe; GLM VQ lane verified on the reference Spark 2026-10-17 per repo README table). Weights: Baekpica/GLM-5.3-Flash-VQ-GGUF @ 4da527cd0f9e83ccd9fb451a1feaf3ba861f7c77 (GVQ1 Q4_K nonexpert v3; 56 parts + tables verified via HF API this ingest; SHA256SUMS verified by the engine at load per runbook). Engine: Baekpica/ds4-dfm-rs (Rust host of antirez/ds4), main from c5384b5e PR86; measured on optimize4 checkout. Not verified upstream: long-context behaviour at 1M, predictor benefit, vision tower (launcher serves text only).

Revision history

Fork

Sign in to fork this recipe — a fork is your own copy; you own it and only you can update it.

Export

recipe.yaml (raw)