GLM-5.3-Flash NVFP4 on 2x DGX Spark - Atlas RC4.2 TP2, DFlash2, 1M

serve /models/atlas-overlay --kernel-target=glm-5.3-flash --world-size=2 --tp-size=2 --max-seq-len=1048576 --gpu-memory-utilization=0.88 --ep-size=2 --max-prefill-tokens=8192 --max-batch-size=4 --max-num-seqs=4 --oom-guard-mb=4096 --kv-cache-dtype=fp8_g128 --ssm-h-dtype=f32 --ssm-rollback-mode=records --block-size=16 --ssm-cache-slots=16 --enable-prefix-caching --ssm-checkpoint-interval=0 --gpu-ordinal=0 --swap-space-gb=0 --no-auto-swap --no-tui --lm-head-dtype=bf16 --request-timeout=0 --video-allow-ffmpeg --vision-allow-remote-images --tool-call-parser=poolside_v1 --video-max-frames=32 --dflash --draft-model=/models/drafter --dflash-gamma=8 --dflash-window-size=2048 --mtp-gate=force

env: "Install layer (install/, image tag 246a3a948ab9 = git tree @ 3c7d671): profile 4x1m environment arms the full RC4.2 decode stack — NCCL Ring/Simple over the CX-7 (NCCL_IB_HCA + NCCL_SOCKET_IFNAME set per node from the detected fabric port, FABRIC_SIBLINGS=1 adds the RoCE device behind the cable's other PCIe domain), ATLAS_RDMA_ALLREDUCE=1 + ATLAS_RDMA_ONESHOT=1 + ATLAS_RDMA_ONESHOT_SM_STAGE=1 + ATLAS_RDMA_PAIR_CHAIN=1 + ATLAS_GLM_CMD_RDMA=1 for the direct RDMA all-reduce, ATLAS_GLM_KV_SHARD=1 + ATLAS_GLM_KV_SHARD_COMPACT=1 when --kv-shard, DFlash2 (ATLAS_GLM_DFLASH=1, ATLAS_DFLASH_OPTION_B=1, ATLAS_DFLASH_ADAPTIVE_WIDTH=1 with ATLAS_DFLASH_CONF_WIDTH=1 + ATLAS_DFLASH_CONF_PROBE=256, ATLAS_DFLASH2_EDGE=0.6, ATLAS_DFLASH_MULTI_DRAFT_CAP=7, ATLAS_GLM_DRAFT_TP=1 + ATLAS_GLM_DRAFT_TP_BATCH=1 + ATLAS_GLM_DRAFT_TP_CTX=1 + ATLAS_GLM_DRAFT_TP_TOPK=1, ATLAS_DFLASH_ASYNC_UPLOADS=1, ATLAS_DFLASH_MXFP8=1 + ATLAS_DFLASH_NVFP4_HEAD=1 + ATLAS_DFLASH_NVFP4_TC=1 + ATLAS_DFLASH_CONV_TWIN=nvfp4), canonical verify (ATLAS_GLM_CANONICAL_VERIFY-class ATLAS_GLM_STRICT_SPEC=1 for structured outputs, ATLAS_GLM_VERIFY_VOCAB_SPLIT=1, ATLAS_GLM_LONG_BATCH_VERIFY=1), prefix cache (ATLAS_GLM_PC_EVICT=1 + ATLAS_GLM_PC_BRANCH=1, Marconi prefill-only ATLAS_MARCONI_PREFILL_ONLY=1, ATLAS_GLM_WARM_SKIP_CACHED=1 + ATLAS_GLM_WARM_CHUNK_RUN=1), NVFP4 kernels (ATLAS_NVFP4_FUSED_SILU_QUANT=1, ATLAS_NVFP4_PREQUANT_MOE=1, ATLAS_NVFP4_VECSCALE=1, ATLAS_UNIFIED_MOE_LAYOUT=1) and the RC4 fusions (ATLAS_GLM_MOE_ROUTE_FUSE=1, ATLAS_GLM_MLA_FUSE=1, ATLAS_GLM_DENSE_TP_SPLIT=1, ATLAS_GLM_KDA_COMMIT_CTAS=96, ATLAS_GLM_MXFP8_TILED=1, ATLAS_PDL=1) — ~220 ATLAS_*/NCCL_* keys total, transcribed in full into install/profiles/4x1m.json @ 3c7d671 (values win over ambient flags: serve.py strips ATLAS_* from the container env and re-applies the profile's own). Ranks: docker run --network host --ipc host --shm-size 32g --memory 114g --gpus all --device /dev/infiniband --cap-add IPC_LOCK --cap-add SYS_NICE --ulimit memlock=-1:-1 --security-opt no-new-privileges=true (start-node.sh); --cap-add SYS_ADMIN only with DISPLAY_CARVEOUT=1. NVMe tier env (serve.py disk_prefix_cache): ATLAS_KV_NVME_DIR=/prefix-cache/kv + ATLAS_KV_NVME_GB=<half>, ATLAS_GLM_NVME_FAST=1, ATLAS_SSM_TIER=1 + ATLAS_SSM_TIER_UNIFIED=1 + ATLAS_SSM_TIER_SWAP_DIR=/prefix-cache/ssm + ATLAS_SSM_TIER_DISK_GB=<half> + ATLAS_SSM_TIER_SLOTS=2."
notes: "SparkGLM RC4.2 (Enntity/sparkglm @ 3c7d671, RC4.2 merge): GLM-5.3-Flash NVFP4 on two DGX Sparks joined by one 200G ConnectX-7 cable, served by the Atlas engine (Enntity/atlas sparkglm/atlas-20261012-rc4.2-layered @ dfb9c26a, tree 89e392a3) with tensor + expert parallelism over RDMA (RoCE v2, both PCIe domains of the cable), DFlash2 speculative decoding (gamma 8, window 2048), FP8-latent shared KV pool with prefix caching, up to 4 concurrent requests with per-request 1M context (4x1m profile), OpenAI-compatible API on 127.0.0.1:8893. ./start.sh on rank 0 drives everything: pulls/builds the image (tag = git tree of install/), downloads both pinned checkpoints once (207 GB) and mirrors them over the cable, converts the 864 NVIDIA MTP matrices to NVFP4 in a verified overlay per node (install/convert.sh, ~1 min), then starts rank 1 over ssh and rank 0 and waits for health (~2.5 min). Optional knobs (default off unless set in .env): DISPLAY_CARVEOUT=1 lends each GB10's 2 GiB display carveout to the KV pool (server drops CAP_SYS_ADMIN after export); PREFIX_CACHE_DIR(+PREFIX_CACHE_GB, 16-100 GiB per node) keeps evicted prefix-cache entries on each node's disk; KV_SHARD=1 (start-node --kv-shard) splits the MLA latent KV across the ranks (~1.8x pool); SPARKGLM_GPU_MEMORY_UTILIZATION 0.80-0.95 overrides the profile's 0.88. Decode is concurrency-invariant (canonical verify): greedy output at c4 equals c1 4/4. Rerun ./start.sh to restart; ./start.sh stop|status|logs|build|download|convert."

Fetch https://localairecipes.com/api/v1/recipes/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m.json