GLM-5.3-Flash on 4x RTX PRO 6000 Blackwell — Karmic Kraken TP4 DFlash2 K7

serve local-inference-lab/GLM-5.3-Flash-NVFP4 --additional-config ''\''{"glm53_kda_decode_backend":"auto","kda_prefill_backend":"b12x"}'\''' --attention-backend B12X --block-size 256 --compilation-config ''\''{"cudagraph_mode":"FULL_AND_PIECEWISE"}'\''' --cp-kv-cache-interleave-size 4 --cudagraph-capture-sizes 1 2 4 8 16 32 40 48 64 96 128 192 256 --dcp-kv-cache-interleave-size 4 --decode-context-parallel-size 1 --default-chat-template-kwargs ''\''{"reasoning_effort":"high","clear_thinking":false}'\''' --dtype bfloat16 --enable-auto-tool-choice --enable-chunked-prefill --no-enable-flashinfer-autotune --enable-prefix-caching --enable-prompt-tokens-details --gpu-memory-utilization 0.93 --host 0.0.0.0 --kv-cache-dtype fp8 --linear-backend b12x --load-format instanttensor --mamba-cache-mode align --max-cudagraph-capture-size 256 --max-model-len 1048576 --max-num-batched-tokens 4096 --max-num-seqs 32 --max-parallel-prefills 1 --moe-backend b12x --override-generation-config ''\''{"temperature":1.0,"top_p":0.95}'\''' --pipeline-parallel-size 1 --port 8000 --prefill-compute-share 0.4 --prefill-schedule-interval 1 --quantization modelopt_mixed --reasoning-parser glm45 --recurrent-checkpoint-policy request_boundaries --served-model-name GLM-5.3-Flash-NVFP4 --speculative-config ''\''{"method":"dflash","model":"local-inference-lab/GLM-5.3-Flash-DFlash2","draft_sample_method":"probabilistic","rejection_sample_method":"standard","attention_backend":"FLASH_ATTN","kv_cache_dtype":"auto","num_speculative_tokens":7}'\''' --tensor-parallel-size 4 --tool-call-parser glm47

Fetch https://localairecipes.com/api/v1/recipes/glm-5-3-flash-4x6000-kk-dflash2-k7.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/glm-5-3-flash-4x6000-kk-dflash2-k7.json