DeepSeek V4 Flash Vision on 2x RTX PRO 6000 Blackwell \x{2014} TP2 DSpark K3

community Community submission — verify on your own hardware.

recipe by local-inference-lab↗ · listed by localai-recipes

revision r2 · commit 17fd5669c2 · updated 10 d ago · license CC-BY-4.0

chat code vision vision dspark jovian-judgement

Community image TP2 DSpark K3 Vision serving recipe: B12X sparse MoE, FP8 KV, three draft layers, 1M context. Saved numbers from the KK comparison.

Launch command

docker compose -f docker-compose-ds4-vision-jovian-judgement-r9.yml up -d

Use this recipe with your agent

Paste this into any agent chat — it fetches and applies the recipe:

Fetch https://localairecipes.com/api/v1/recipes/ds4-vision-2x6000-jj-dspark-k3.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

Or fetch it directly:

curl -fsSL https://localairecipes.com/api/v1/recipes/ds4-vision-2x6000-jj-dspark-k3.json

Engine

vllm · source↗ · voipmonitor/vllm:jovian-judgement-vllmf66599d-b12x15b6813-fi803c466-cu133-torch213-20260907-r9↗

Model

DeepSeek-V4-Flash-Vision-Exp (fp8) · deepseek-ai/DeepSeek-V4-Flash-Vision-Exp↗
AA intelligence 35

Hardware

NVIDIA RTX PRO 6000 Blackwell 96GB ×2

Performance

Speed by context

Context	Conc.	Prefill	Decode	TTFT
1	1	—	169.2 tok/s	—	reported
1	4	—	394.7 tok/s	—	reported
32,768	1	8993 tok/s	—	—	reported

prefill = prompt processing · decode = generation. All decode/prefill figures are shown as reported.

Install requirements

Docker (or a compatible OCI container runtime)
NVIDIA Container Toolkit (nvidia-container-toolkit) - exposes host GPUs to the container

NVIDIA driver floor: NVIDIA driver >= 580 (image CUDA 13.3 - CUDA 13.x applications run on drivers >= 580 under CUDA minor-version compatibility; the CUDA 13.3 toolkit shipped driver >= 610.43.02)

Source: NVIDIA CUDA Toolkit Release Notes - driver compatibility tables (https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/, fetched 2026-10-03).

Map the placeholders to your rig

Hardware-bound values in this recipe (device counts, bind-mount paths, utilization, ports) describe the author's machine, not yours - map them to your own rig before launching. Keep the engine flags, model, quant, and parallelism strategy; adjust paths and ports to the hardware you actually have.
GPU identity: list your GPUs with `nvidia-smi -L` and substitute your own GPU UUIDs/indices wherever the recipe pins the author's.
Model directory: bind-mount paths are author-side - point them at your own mount where the model files live.
Host port: pick a free port on your machine and publish the container there; keep the in-container port unchanged.

Revision notes

Vision compose: MODE vision profile, request slots 4, graph cap 16, speculative depth > 3 unsupported.

Revision history

bfdf9daa3e history: re-squash corpus to a single root commit after the slug reconciliation (prior tip kept at refs/lar-archive/pre-single-slug-resquash) 25 d ago

Fork

Export

recipe.yaml (raw)