Beta branch — the upstream image for this build is a beta and is being superseded (see the recipe notes); measured numbers may shift between image refreshes.

DeepSeek V4.1 Flash on 4x RTX PRO 6000 Blackwell — Karmic Kraken TP4 adaptive DSpark K7

0 sign in to vote

community Community submission — verify on your own hardware.

recipe by local-inference-lab↗ · listed by localai-recipes

revision r2 · commit 7dbac8c7cc · updated 10 d ago · license CC-BY-4.0

chat code adaptive

Karmic Kraken TP4 serving recipe: adaptive-verification DSpark K7, RAM Engram, 131K context (4.7M logical KV tokens).

Launch command

serve deepseek-ai/DeepSeek-V4.1-Flash --async-scheduling --attention-backend B12X --block-size 256 --compilation-config ''\''{"cudagraph_mode":"FULL_AND_PIECEWISE"}'\''' --decode-context-parallel-size 1 --decode-refill-target auto --default-chat-template-kwargs ''\''{"thinking":true,"reasoning_effort":"high"}'\''' --dtype bfloat16 --enable-auto-tool-choice --enable-chunked-prefill --enable-force-include-usage --enable-prefix-caching --enable-prompt-tokens-details --enable-request-id-headers --engram-config ''\''{"cpu_offload":false,"table_memory":"disk","disk_resident_scales":false,"projection_tp":false}'\''' --generation-config vllm --gpu-memory-utilization 0.95 --host 0.0.0.0 --jit-monitor-mode warn --kv-cache-dtype fp8 --linear-backend b12x --load-format instanttensor --max-cudagraph-capture-size 128 --max-model-len -1 --max-num-batched-tokens 4096 --max-num-seqs 32 --max-parallel-prefills 1 --moe-backend b12x --override-generation-config ''\''{"temperature":1.0,"top_p":0.95}'\''' --pipeline-parallel-size 1 --port 8000 --prefill-compute-share 0.4 --prefill-policy round-robin --prefill-schedule-interval 1 --reasoning-parser deepseek_v41 --safetensors-load-strategy lazy --no-scheduler-reserve-full-isl --served-model-name DeepSeek-V4.1-Flash --speculative-config ''\''{"method":"dspark","attention_backend":"B12X","draft_sample_method":"greedy","rejection_sample_method":"standard","num_speculative_tokens":7,"draft_tensor_parallel_size":4,"enable_adaptive_verification":true,"adaptive_verification_cost_scale":1.0}'\''' --swa-block-size 128 --tensor-parallel-size 4 --tokenizer-mode deepseek_v41 --tool-call-parser deepseek_v41

Use this recipe with your agent

Paste this into any agent chat — it fetches and applies the recipe:

Fetch https://localairecipes.com/api/v1/recipes/ds4-1-flash-4x6000-kk-dspark-k7.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

Or fetch it directly:

curl -fsSL https://localairecipes.com/api/v1/recipes/ds4-1-flash-4x6000-kk-dspark-k7.json

Engine

vllm · source↗ · cfdecfaa↗ · ghcr.io/local-inference-lab/vllm:karmic-kraken-beta-20260922-3221ccacf71002ea↗

Model

DeepSeek-V4.1-Flash · deepseek-ai/DeepSeek-V4.1-Flash↗
AA intelligence 39 (Artificial Analysis · measured)

Hardware

NVIDIA RTX PRO 6000 Blackwell 96GB ×4

Install requirements

Docker (or a compatible OCI container runtime)
NVIDIA Container Toolkit (nvidia-container-toolkit) - exposes host GPUs to the container

NVIDIA driver floor: unknown - check the image tag

Source: NVIDIA CUDA Toolkit Release Notes - driver compatibility tables (https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/, fetched 2026-10-03).

Map the placeholders to your rig

Hardware-bound values in this recipe (device counts, bind-mount paths, utilization, ports) describe the author's machine, not yours - map them to your own rig before launching. Keep the engine flags, model, quant, and parallelism strategy; adjust paths and ports to the hardware you actually have.
GPU identity: list your GPUs with `nvidia-smi -L` and substitute your own GPU UUIDs/indices wherever the recipe pins the author's.
Model directory: bind-mount paths are author-side - point them at your own mount where the model files live.
Host port: pick a free port on your machine and publish the container there; keep the in-container port unchanged.

Revision notes

Benchmark-qualified configuration from local-inference-lab/rtx6kpro (Karmic Kraken serving report). Five warmed 30-second windows per cell on RTX PRO 6000 Blackwell Max-Q; decode ctx-0 temp 1. Launch argv transcribed verbatim from the repo's unified Docker guide (same composition). Measured: C1 249.1 tok/s, C8 805.5 tok/s, 32K prefill 17982 tok/s.

Revision history

bfdf9daa3e history: re-squash corpus to a single root commit after the slug reconciliation (prior tip kept at refs/lar-archive/pre-single-slug-resquash) 25 d ago

Fork

Export

recipe.yaml (raw)