Motif-3 NVFP4 314B 2x DGX Spark TP2 vLLM Fork FP8 Dense MTP

'# launch authority AT 91ee567c61229cbe75536901053f0061d0fc8835: scripts/node.sh (head: node.sh 0; worker: node.sh 1, normally started by scripts/start.sh on the head over ssh)' 'docker run -d --name motif3 --gpus all --network host --ipc host --shm-size 16g --device /dev/infiniband --ulimit memlock=-1 --ulimit stack=67108864 --cap-add IPC_LOCK --security-opt seccomp=unconfined --memory 112g' '-v $HOME/.cache/huggingface:/root/.cache/huggingface:ro -v $HOME/.cache/motif3/vllm:/root/.cache/vllm -v $HOME/.cache/motif3/triton:/root/.triton -v $HOME/.cache/motif3/flashinfer:/root/.cache/flashinfer -v $HOME/.cache/motif3/tilelang:/root/.tilelang -v $HOME/.cache/motif3/inductor:/tmp/torchinductor_root -v $HOME/.cache/motif3/nv:/root/.nv' '-e HF_HUB_OFFLINE=1 -e HF_HUB_DISABLE_XET=1 -e HF_MODULES_CACHE=/root/.cache/vllm/hf_modules -e MOTIF_MHC_TILELANG=0 -e LD_LIBRARY_PATH=/usr/local/cuda/compat/lib.real:/usr/local/lib/python3.12/dist-packages/torch/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64' '-e VLLM_HOST_IP=$IP (head HEAD_IP=192.168.0.1 / worker WORKER_IP=192.168.4.1) -e NCCL_SOCKET_IFNAME=enp1s0f0np0 -e GLOO_SOCKET_IFNAME=enp1s0f0np0' '-e NCCL_IB_HCA=rocep1s0f0,rocep1s0f1,roceP2p1s0f0,roceP2p1s0f1 -e NCCL_IB_SUBNET_AWARE_ROUTING=1 -e NCCL_IB_MERGE_NICS=0 -e NCCL_NET_PLUGIN=none -e NCCL_NET=IB -e NCCL_IGNORE_CPU_AFFINITY=1 ${EXTRA_ENV:-}' motif-vllm:0.20.2-motif3-sm121-r5 'vllm serve /root/.cache/huggingface/hub/models--Motif-Technologies--Motif-3-NVFP4/snapshots/79f2fad1f8229f5db8a7ad08bbd24a20d6fb0bff --served-model-name Motif-Technologies/Motif-3-NVFP4 --trust-remote-code' '--quantization modelopt_nvfp4 --dtype bfloat16 --tensor-parallel-size 2 --nnodes 2 --node-rank $RANK --master-addr 192.168.0.1 --master-port 29511' '--gpu-memory-utilization 0.80 --max-model-len 262144 --max-num-seqs 4 --block-size 128' '--enable-auto-tool-choice --tool-call-parser motif --reasoning-parser motif' '--speculative-config '\''{"model": "/root/.cache/huggingface/hub/models--Motif-Technologies--Motif-3-NVFP4/snapshots/79f2fad1f8229f5db8a7ad08bbd24a20d6fb0bff", "num_speculative_tokens": 1}'\'' (MTP=1; MTP=0 drops this flag)' '# head adds: --host 127.0.0.1 --port 8000; worker adds: --headless (worker started via ssh spark-2-cl '\''node.sh 1'\'')'

- HEAD_IP=192.168.0.1
- WORKER_IP=192.168.4.1
- WORKER_SSH=spark-2-cl
- API_HOST=127.0.0.1
- API_PORT=8000
- NET_IF=enp1s0f0np0
- 'IB_HCA=rocep1s0f0,rocep1s0f1,roceP2p1s0f0,roceP2p1s0f1'
- 'IMG=motif-vllm:0.20.2-motif3-sm121-r5'
- REV=79f2fad1f8229f5db8a7ad08bbd24a20d6fb0bff
- MAXLEN=262144
- GPU_UTIL=0.80
- MTP=1
- MAX_SEQS=4
- HF_HUB_OFFLINE=1
- HF_HUB_DISABLE_XET=1
- HF_MODULES_CACHE=/root/.cache/vllm/hf_modules
- MOTIF_MHC_TILELANG=0
- 'LD_LIBRARY_PATH=/usr/local/cuda/compat/lib.real:/usr/local/lib/python3.12/dist-packages/torch/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64'
- NCCL_SOCKET_IFNAME=enp1s0f0np0
- GLOO_SOCKET_IFNAME=enp1s0f0np0
- 'NCCL_IB_HCA=rocep1s0f0,rocep1s0f1,roceP2p1s0f0,roceP2p1s0f1'
- NCCL_IB_SUBNET_AWARE_ROUTING=1
- NCCL_IB_MERGE_NICS=0
- NCCL_NET_PLUGIN=none
- NCCL_NET=IB
- NCCL_IGNORE_CPU_AFFINITY=1

notes: "Rank 0 (head) is the API node, rank 1 is --headless; scripts/start.sh ssh-forwards the knobs and starts both (node.sh IS the launch authority; README launch text is prose). MOTIF_MHC_TILELANG=0 is REQUIRED: the default mHC path calls DeepGEMM which is SM90/SM100 only. LD_LIBRARY_PATH must start with /usr/local/cuda/compat/lib.real because the image ships CUDA 13.1 while the Spark driver is CUDA 13.0 and FA2 is shipped as sm_80 PTX (its JIT fails with cudaErrorUnsupportedPtxVersion otherwise). HF_MODULES_CACHE must be on a writable path (the HF cache mount is read-only). The fork forces expert parallel (192 of 384 experts per node); on SM12x MLA runs on TRITON_MLA so CUDA graphs are PIECEWISE only. --network host with API_HOST bound explicitly - do not use 0.0.0.0 on a shared LAN. FP8 dense layer is off with EXTRA_ENV='-e MOTIF_FP8_DENSE=0 -e MOTIF_FP8_LM_HEAD=0' (costs 15-17.6 vs ~19 tok/s). Thinking is controlled per request via chat_template_kwargs.enable_thinking plus thinking_token_budget (server default 80% of remaining space)."

Fetch https://localairecipes.com/api/v1/recipes/motif-3-nvfp4-314b-2x-dgx-spark-tp2-vllm-fork-fp8-dense-mtp.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/motif-3-nvfp4-314b-2x-dgx-spark-tp2-vllm-fork-fp8-dense-mtp.json