DeepSeek-V4.1-Flash on 4x DGX Spark — tuned TP4 SGLang/DSpark

python3 -u /opt/dsv41/boot.py run --model-path /models/DeepSeek-V4.1-Flash --served-model-name deepseek-v4.1-flash --trust-remote-code --load-format safetensors --tp 4 --ep-size 1 --nnodes 4 --node-rank 0 --dist-init-addr 10.0.0.1:20000 --attention-backend dsv4 --moe-runner-backend flashinfer_mxfp4 --mem-fraction-static 0.80 --chunked-prefill-size 4096 --context-length 1048576 --max-running-requests 16 --cuda-graph-max-bs-decode 16 --max-total-tokens 8000000 --random-seed 0 --enable-decoder-swa-bounded-replay --speculative-algorithm DSPARK --speculative-dspark-block-size 5 --tool-call-parser deepseekv41 --reasoning-parser deepseek-v41 --host 0.0.0.0 --port 8888 --fp8-gemm-backend flashinfer_cutlass --watchdog-timeout 1800 --min-free-slots-delay 1 --enable-deepseek-v4-fp4-indexer --enable-cache-report --sleep-on-idle --model-loader-extra-config '{"num_threads":1}'

env: 'EXTRA_CONTAINER_ENV (production v2.3 line, mirrored to every rank): DSV41_INDEXER_CHUNKED=1 SGLANG_DSPARK_FOLDED_SAMPLING=2 SGLANG_ROCE_ALLREDUCE=1 SGLANG_ROCE_MAX_SIZE=2097152 B12X_ROCE_HCA=rocep1s0f0,roceP2p1s0f0 B12X_ROCE_CACHE_DIR=/state/b12x-roce B12X_COMPILE_CACHE_DIR=/state/b12x-compile SPARK_PREFILL_TP_SPLIT=1 SPARK_PREFILL_TP_MIN_CONTEXT=32768 SPARK_PREFILL_TP_MIN_ROWS=1024 SGLANG_RUST_BUILD_MODE=never DSV41_FAST_LOAD=1 DSV41_FAST_LOAD_EP_SIZE=1 DSV41_FAST_LOAD_TP_SLICE=auto DSV41_FAST_LOAD_N_EXPERTS=384 DSV41_FAST_LOAD_INFLIGHT_GB=6 DSV41_ENGRAM_PREFETCH=1 DSV41_SHARED_PAD_K=1 DSV41_WO_A_W8=1 DSV41_DRAFT_TAU=0.7 DSV41_DRAFT_HEAD_FP8=1 DSV41_BLOCK_VERIFY=1 DSV41_FOLDED_FENCE=1 DSV41_VERIFY_CAP=conf:0.1 DSV41_AUTOTUNE_KEEP=1 DSV41_WO_A_W8_MID=1 DSV41_WO_A_W8_DROP=1 DSV41_REPLICATED_SPLIT=wqkv_a,engram.wkv DSV41_DRAFT_MAIN_PROJ_SPLIT=1 DSV41_ROCE_GATHER=2097152 DSV41_ROUTER_LIVE=1 DSV41_MOE_B12X_NEXT=1 DSV41_MOE_B12X_NEXT_DETERMINISTIC=1 DSV41_HC_FUSED=1 DSV41_PREFILL_SP=1 DSV41_PREFILL_SP_FP8=1 DSV41_L2_PREFETCH=1 DSV41_L2_PREFETCH_WOA=1 DSV41_SPEC_SYNC_FREE=all DSV41_EAGER_GLUE=all DSV41_SPLIT_COMPACT_GATHER=1 DSV41_CERT_HEAD=1 DSV41_CERT_HEAD_M=6,12,18,24,30,36,42,48 | Also per .env.tp4.example: EP_SIZE=1 DSV41_MXFP8_BACKEND=b12x DSV41_SHARED_PAD_K=1 DSV41_CACHE_GIB=4 DSV41_CACHE_WAYS=16 DSV41_IO_THREADS=96 DSV41_ENGRAM_PREFETCH=1 DSV41_TP_PAD=0 DSV41_MAX_NEW_TOKENS=32768 DSV41_LOOP_ABORT=1 SGLANG_DSV41_REASONING_EFFORT=75 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False DSV41_PREFILL_EMPTY_CACHE_TOKENS=8192 SGLANG_FLASHINFER_MOE_FUSED_FINALIZE=0 NCCL_BUFFSIZE=1048576 NCCL_PROTO=^LL128 NCCL_MAX_NCHANNELS=8 B12X_ROCE_HCA=rocep1s0f0,roceP2p1s0f0 | Launch: IMAGE=dsv41-4x-spark:canary-roce, BUILD_DOCKERFILE=Dockerfile.canary-roce, EP_SIZE=1, port 8888 + dist port 20000 + NFS 2049/111 node-to-node.'
notes: 'Run ./start-tp4.sh (doctor|download|build|share|pack|serve): boot.py profile bootstrap, .env.tp4 from .env.tp4.example (HEAD_IP/WORKER_IPS/MODEL_DIR/fabric), dockerized 4-rank fleet --gpus all --privileged --ipc host, ranks 1-3 over ssh, head rank 0 on :8888; boot log must show the DSV41 arming lines (shared-expert padding, indexer chunked, DSpark gamma=5, max_running_requests=16, RoCEnante ready, moe_b12x_next armed EP1, hc_fused bit-identical). Weights deepseek-ai/DeepSeek-V4.1-Flash @ fb2764a5 (476 GiB, 48 shards) at the pinned revision, packed Engram rows ~48 GiB NVMe/node (./start-tp4.sh pack). Fabric: switched RoCE (both CX-7 rails rocep1s0f0,roceP2p1s0f0) for the production line; switchless ring is opt-in (docs/switchless-ring.md, DSV41_ROCE_RING + patched NCCL). Boot ~556 s cold / ~120 s with DSV41_FAST_LOAD; KV pool 6.8-7.3M tokens at 1M context; MEM_FRACTION_STATIC=0.80 keeps ~24 GiB/rank outside the static pool for the bounded prefill transient (indexer chunked 2 GiB rows, sglang#39187 backport). MEASURED FABRIC CAVEAT: headline numbers are on a switched RoCE fabric; a switchless ring without opposite-node hardware paths falls back to NCCL and decodes slower, and ring prefill is lower either way (one-link bisection). Known limits: prompt-token logprobs unavailable (--enable-decoder-swa-bounded-replay, requests 400, DSV41_REPLAY_GUARD), single-stream prose bounded by DSpark acceptance ~3 tok/step, fast loader leaves KV pool 3-13% smaller. Rollback: every layer is an env change (see README Rollback table); upstream MiaAI profile = MAX_RUNNING_REQUESTS=8, CHUNKED_PREFILL_SIZE=1024, EP_SIZE=4, EXTRA_CONTAINER_ENV unset.'

Fetch https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-on-4x-dgx-spark-tuned-tp4-sglang-dspark.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-on-4x-dgx-spark-tuned-tp4-sglang-dspark.json