DeepSeek-V4.1-Flash-1017 Fidelity-First 4x CMP 170HX alanoo81

'docker run -d --name ds41-flash --device nvidia.com/gpu=all --ipc=host --cap-add SYS_NICE' '--ulimit memlock=-1 --ulimit stack=67108864' '-v <MODELS_DIR>:/models:ro -v /opt/engram-ram/DeepSeek-V4.1-Flash-1017-engram:/engram:ro' '-v <CACHE_DIR>/root-cache:/root/.cache -v <CACHE_DIR>/triton:/root/.triton -v <CACHE_DIR>/tilelang:/root/.tilelang' '-p 127.0.0.1:8011:8011' '-e VLLM_PLUGINS=vllm_exl3 -e VLLM_ENGINE_READY_TIMEOUT_S=3600 -e VLLM_EXL3_MODEL_DIR=/models/DeepSeek-V4.1-Flash-1017-hybrid' '-e VLLM_USE_V2_MODEL_RUNNER=0 -e DSV41_DSPARK_V1=1 -e VLLM_EXL3_COOP=1' '-e VLLM_EXL3_TC_MOE_V2=1 -e VLLM_EXL3_TC_MOE_PREFILL=1 -e VLLM_EXL3_TC_PREFILL_ROWS=2048' '-e VLLM_EXL3_TC_MOE_SO=/opt/dsv41-tc-moe/_dsv41_tc_moe_C.abi3.so -e VLLM_EXL3_TC_HOOK=/opt/dsv41-tc-moe/exl3_tc_hook.py' '-e VLLM_EXL3_MOE_TP_ALIGN=128 -e VLLM_EXL3_MOE_TP_ROTATE=1' '-e NCCL_CUMEM_ENABLE=0 -e NCCL_CUMEM_HOST_ENABLE=0 -e NCCL_P2P_LEVEL=SYS -e VLLM_ALLREDUCE_USE_FLASHINFER_PCIE_IPC=1' '-e DSV41_ENGRAM_DIR=/engram -e DSV41_ENGRAM_STORAGE=ssd' '-e DSV41_EXPERTS_ONLY_FILES_RE=^model-[0-9]+-of-000(43|44|39)\.safetensors$' '-e TORCH_CUDA_ARCH_LIST=8.0 -e TORCHINDUCTOR_CACHE_DIR=/root/.cache/inductor' '-e VLLM_DSV41_DECODE_SPLITK=1 -e DSV41_SHARED_HEAVY_BLOCKS=1 -e DSV41_DRAFT_HEAD_FP8=1' '-e DSV41_PREFILL_CHUNK_REQS=1 -e DSV41_INDEXER_BUF_MULT=4 -e VLLM_EXL3_FAT_THRESHOLD=2048' '--entrypoint vllm ds41-cmp170hx:1.2.0-fidelity serve /models/DeepSeek-V4.1-Flash-1017-hybrid' '--quantization exl3 --load-format safetensors --safetensors-load-strategy lazy' '--tensor-parallel-size 4 --pipeline-parallel-size 1' '--served-model-name deepseek-v4.1-flash --host 0.0.0.0 --port 8011' '--max-model-len 262144 --max-num-seqs 8 --max-num-batched-tokens 2048 --gpu-memory-utilization 0.98' '--kv-cache-dtype fp8_ds_mla --enable-prefix-caching' '--compilation-config {"cudagraph_mode": "FULL_AND_PIECEWISE", "cudagraph_capture_sizes": [1,2,3,4,5,6,8,12,16,18,20,24,28,30,32,36,42,48]}' --async-scheduling '--tokenizer-mode deepseek_v41 --enable-auto-tool-choice --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41' --enable-prompt-tokens-details

P2P: 1
BIND: 127.0.0.1
PORT: 8011
IMAGE: 'ds41-cmp170hx:1.2.0-fidelity'
LAYOUT: tp4
EXTRA_ARGS: '--enable-prompt-tokens-details'
SPEC_TOKENS: 0
MAX_NUM_SEQS: 8
MAX_MODEL_LEN: 262144
ENGRAM_STORAGE: ssd
ENGRAM_HOST_DIR: /opt/engram-ram/DeepSeek-V4.1-Flash-1017-engram
GPU_MEMORY_UTILIZATION: 0.98
VLLM_EXL3_FAT_THRESHOLD: 2048

notes: "Patched serve.sh: SPEC_TOKENS=0 drops --speculative-config entirely (no DSpark drafter, no DSV41_DYNK); ENGRAM_HOST_DIR mounted read-only at /engram with DSV41_ENGRAM_STORAGE=ssd; start.sh runs engram-ram.sh first when ENGRAM_HOST_DIR is under /opt/engram-ram/ (tmpfs ENGRAM_TMPFS_SIZE=200G, copy ~94 s from NVMe). API has no auth by default - bind stays 127.0.0.1 until EXTRA_ARGS carries --api-key. Startup log must show 'Built engram token map (129280 -> 99092 ids) for layers (1, 14)' per rank and NO 'offloaded to pinned' line; 'GPU KV cache size: 2,717,420 tokens' at 0.98 (2,417,392 at 0.97, 1,817,233 at 0.95). Graft build: tools/make_hybrid_graft.py k3 = 188,241 tensors, 23,040 grafted at 3 bits, 88 links."

Fetch https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-fidelity-first-4x-cmp-170hx-alanoo81.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-fidelity-first-4x-cmp-170hx-alanoo81.json