DeepSeek V4 Flash Vision on 2x RTX PRO 6000 Blackwell — Karmic Kraken TP2 DSpark K3

serve deepseek-ai/DeepSeek-V4-Flash-Vision-Exp --async-scheduling --attention-backend B12X --block-size 256 --code-revision 6821d6ad3681a4b137b066b76094fa82ebd0a380 --compilation-config ''\''{"cudagraph_mode":"FULL_AND_PIECEWISE","custom_ops":["all"]}'\''' --decode-context-parallel-size 1 --default-chat-template-kwargs ''\''{"thinking":true,"reasoning_effort":"high"}'\''' --dtype bfloat16 --enable-auto-tool-choice --enable-chunked-prefill --enable-flashinfer-autotune --enable-force-include-usage --enable-prefix-caching --enable-prompt-tokens-details --enable-request-id-headers --gpu-memory-utilization 0.975 --host 0.0.0.0 --kv-cache-dtype fp8 --load-format instanttensor --max-cudagraph-capture-size 16 --max-model-len -1 --max-num-batched-tokens 4096 --max-num-seqs 4 --moe-backend b12x --override-generation-config ''\''{"temperature":1.0,"top_p":0.95}'\''' --pipeline-parallel-size 1 --port 8000 --prefix-cache-retention-interval 4096 --reasoning-parser deepseek_v4 --revision 6821d6ad3681a4b137b066b76094fa82ebd0a380 --no-scheduler-reserve-full-isl --served-model-name DeepSeek-V4-Flash-Vision-Exp --speculative-config ''\''{"method":"dspark","draft_sample_method":"probabilistic","rejection_sample_method":"standard","num_speculative_tokens":3,"model":"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp","revision":"6821d6ad3681a4b137b066b76094fa82ebd0a380"}'\''' --tensor-parallel-size 2 --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 --trust-remote-code

Fetch https://localairecipes.com/api/v1/recipes/ds4-vision-2x6000-kk-dspark-k3.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/ds4-vision-2x6000-kk-dspark-k3.json