{"slug":"deepseek-v4-1-flash-1017-engram-nvme-tp4-pp4-4x-cmp-170hx","title":"DeepSeek-V4.1-Flash-1017 Engram-NVMe TP4/PP4 4x CMP 170HX","summary":"DeepSeek-V4.1-Flash-1017 (748B MoE) on four CMP 170HX cards power-capped at 180 W, with the model's 189 GiB Engram n-gram tables streamed from NVMe on demand. Recipe 1.2.0 (author measurements, claimed_by=submitter T0): 355.7/258.4/183.6 tok/s one user (structured/code/prose), 867.6/544.0/440.5 across eight, 2,416 tok/s cold prefill (TP4; PP4 2,311), 1,395,731-token KV pool at 262,144 context (1,270,789 at 1M), HumanEval 154/164 (6 length-capped; McNemar p>=0.5 vs 156/164), GSM8K 1,275/1,319 (TP4). Hybrid checkpoint assembled without copying tensors: LibertAI EXL3 2.9-bpw routed experts (layers 0-39) + official FP8 attention/shared experts/head/DSpark drafter/Engram shards + BF16 embeddings/LM head/vision, fp8_ds_mla KV. Engine: vLLM on the community sm_80 stack (zebgop-ops sm80 sparse attention + DSpark V1 + Engram row store, FlashInfer PCIe IPC all-reduce) plus this recipe's measured keep-or-drop patches. TP4 (default, needs x16 + P2P) and PP4 (x1/x4, no P2P) from one .env switch.","license":"MIT; patches/vllm-exl3 AGPL-3.0-only; vendored tc_moe kernels Apache-2.0 (built image AGPL)","tier":"community","beta":false,"revision":{"no":2,"commit_sha":"1b9a60d70c9e1e1f83c7db45556ac3450d511600","content_hash":"dadcb54c6118ad769c07ea0930690a53cead0c65","created_at":"2026-10-28 23:26:24.539707-06","notes":"Community ingest of GNmac-lab's recipe (MIT; patches/vllm-exl3 AGPL-3.0-only as it modifies vcruz305/vllm-exl3; built image contains AGPL-3.0 code, see NOTICE.md). All perf numbers are the author's own measurements (claimed_by=submitter T0; docs/results.md + docs/tuning-log.md include what did NOT help). Hardware note: 180 W/card power caps, 74-SM unlock, PCIe 2.0 x16 with P2P; PP4 lane documented for x1/x4 non-P2P rigs. Announced cmpunlocker #clanker-talk 2026-10-21 13:59Z by greencandle, cross-posted #development-general 14:06Z; Discord numbers match README. NOT a dupe of the sibling deepseek-v4-1-flash-1017-exl3-2-9bpw-tc-moe-on-4x-cmp-170hx-tp4: same hybrid checkpoint approach but a different engine stack (zebgop-ops dsv41reap-pp drop-ins + FlashInfer PCIe IPC all-reduce vs Morrowmake host all-reduce + arena cache), its own measured tables and quality scores, and the site's only 1017 PP4 row. Perf-point context_len markers: TP4 rows base 262144 (+10240 code, +20480 prose; PP4 base +40960); prefill rows use markers 10240 (TP4) / 20480 (PP4) to avoid (ctx,conc,claimed_by) dedup collisions - cold prefill was measured on ~24-38K-token prompts, median of 9. Also credits turboderp (exllamav3 + EXL3 codec) and HalfVulpes (Deepinfer-SM80, credited upstream). Engine watch on ingest: GNmac-lab/deepseek-v41-flash-1017-cmp170hx @ bbaf367291 (pin at ingest; repo created 2026-10-21, expect movement); zebgop-ops/dsv41reap-pp already watched @ 138e7488; LibertAIDAI EXL3 pack already watched @ 0be63c33; deepseek-ai/DeepSeek-V4.1-Flash-1017 already watched @ 1bae0e10. Facts re-verified at pinned HEAD 2026-10-22 - no corrections to the scout card. 1.2.0 PUBLISH (kanban t_d9bb5ffd, owner go 2026-10-28): author release 1.2.0 @ 2ad5741ab1 (both commits since the pin: 45ced96 2026-10-23 = 1.1.0, 2ad5741 2026-10-26 = 1.2.0; the 1.0.0 pin bbaf367291 is superseded). Engine pins added since the 1.0.0 pin: at 1.1.0 - dsv41_fp8_gemv.py, dsv41_fp8w/, dsv41_markov.py, dsv41_woa/, 14 patch_*.py appliers incl. patch_v38 (DSV41_DRAFT_HEAD_FP8), patches/vllm-exl3/, patches/deepinfer_sm80_*; at 1.2.0 - dsv41_dynk.py + patch_dynk.py, eight new variant modules (attn_slices, ce_ring, engram_prefetch, mhc_pcfg, moe_try, moe_v2try, prefill_pack, split_reduce), tc_moe_try/ + tc_moe_v2try/ vendored Apache-2.0 tensor-core kernels. No upstream dependency SHA moved; only the Dockerfile patch ordering grew (21 -> 34 lines). Quality note (docs/results.md:62 @ 2ad5741ab1): HumanEval 154/164 (6 length-capped) is a DOWNGRADE vs the published 156/164; GSM8K unchanged 1,275/1,319. All numbers remain author-measured T0 - no fleet re-bench."},"engine_source":{"type":"docker_image","git_url":"https://github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx","base_engine":"vllm","pip_package":null,"base_version":"patch scripts @ 2ad5741ab1 (built image, ~30-45 min)","image_digest":null,"install_note":"One command from a fresh clone: git clone https://github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx && cd deepseek-v41-flash-1017-cmp170hx && cp .env.example .env && tools/download.sh && ./start.sh - builds the image on first use (~30-45 min), starts, waits until ready; ./start.sh smoke = one chat + one tool call + one image. Model parts + hybrid assembly are resumable. Host: 4x CMP 170HX unlocked to 64 GiB / 74 SMs; TP4 needs PCIe x16 + P2P (tools/p2p_check.sh verifies; LAYOUT=pp4 for x1/x4); NVIDIA driver with CUDA 13 (measured 610.43.02); Docker + NVIDIA Container Toolkit; ~400 GiB fast NVMe for checkpoints + ~40 GiB image; 64 GB+ host RAM (spare RAM becomes page cache for Engram rows). Optional 180 W power cap per card used for every published number. 1.2.0 @ 2ad5741ab1: Dockerfile patch steps 21->34 (dsv41_fp8w/woa ext, dsv41_variants, tc_moe_try builds); tc_moe kernels vendored Apache-2.0 (Morrowmake a0c57b5). Env knobs: config.env_defaults.","pinned_commit":"2ad5741ab1a278d64f33b842706a41717dfbf181"},"credit":[{"name":"GNmac-lab / greencandle (recipe, patches, measurements)","url":"https://github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx"},{"name":"CurtisAccelerate - TP4 hybrid-checkpoint recipe + tensor-core EXL3 MoE port","url":"https://github.com/CurtisAccelerate/DeepSeekv41-Flash-EXL3-TC-MoE"},{"name":"zebgop-ops - dsv41reap-pp sm_80 sparse attention, PP shadow KV, DSpark V1, Engram row store","url":"https://github.com/zebgop-ops/dsv41reap-pp"},{"name":"vcruz305 - vllm-exl3 (AGPL-3.0-only)","url":"https://github.com/vcruz305/vllm-exl3"},{"name":"LibertAI - EXL3 2.9-bpw quantization (MIT)","url":"https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw"},{"name":"DeepSeek - DeepSeek-V4.1-Flash-1017 base model (MIT)","url":"https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017"},{"name":"kaka86mm - dsv41-flash-pp4-170hx (watched sibling)","url":"https://github.com/kaka86mm/dsv41-flash-pp4-170hx"},{"name":"Morrowmake - CMP 170HX recipe format, P2P work, tensor-core MoE kernels","url":"https://github.com/Morrowmake/vllm-cmp170hx"},{"name":"Mia-AiLab - decode benchmark protocol + prompts (used as data)","url":"https://huggingface.co/Mia-AiLab"},{"name":"FlashInfer - PCIe IPC all-reduce","url":"https://github.com/flashinfer-ai/flashinfer"}],"author":{"name":"localai-recipes","isSite":false},"uploaded_by":"localai-recipes","config":{"launch":["docker run -d --name ds41-flash --gpus all --ipc=host --cap-add SYS_NICE --ulimit memlock=-1 --ulimit stack=67108864","-v <MODELS_DIR>:/models:ro -v ./cache/root-cache:/root/.cache -v ./cache/triton:/root/.triton -v ./cache/tilelang:/root/.tilelang","-p 127.0.0.1:8000:8000","-e VLLM_PLUGINS=vllm_exl3 -e VLLM_ENGINE_READY_TIMEOUT_S=3600 -e VLLM_EXL3_MODEL_DIR=/models/DeepSeek-V4.1-Flash-1017-hybrid","-e VLLM_USE_V2_MODEL_RUNNER=0 -e DSV41_DSPARK_V1=1 -e VLLM_EXL3_COOP=1","-e VLLM_EXL3_TC_MOE_V2=1 -e VLLM_EXL3_TC_MOE_PREFILL=1 -e VLLM_EXL3_TC_PREFILL_ROWS=2048","-e VLLM_EXL3_TC_MOE_SO=/opt/dsv41-tc-moe/_dsv41_tc_moe_C.abi3.so -e VLLM_EXL3_TC_HOOK=/opt/dsv41-tc-moe/exl3_tc_hook.py","-e VLLM_EXL3_MOE_TP_ALIGN=128 -e VLLM_EXL3_MOE_TP_ROTATE=1","-e NCCL_CUMEM_ENABLE=0 -e NCCL_CUMEM_HOST_ENABLE=0 -e NCCL_P2P_LEVEL=SYS","-e DSV41_ENGRAM_STORAGE=ssd -e DSV41_ENGRAM_DIR=/models/DeepSeek-V4.1-Flash-1017-engram","-e DSV41_EXPERTS_ONLY_FILES_RE='^model-[0-9]+-of-000(43|39)\\.safetensors$' -e TORCH_CUDA_ARCH_LIST=8.0 -e TORCHINDUCTOR_CACHE_DIR=/root/.cache/inductor","-e VLLM_DSV41_DECODE_SPLITK=1 -e DSV41_SHARED_HEAVY_BLOCKS=1 -e DSV41_DRAFT_HEAD_FP8=1 -e DSV41_PREFILL_CHUNK_REQS=1 -e DSV41_INDEXER_BUF_MULT=4 -e DSV41_DYNK=accept:3,5","-e VLLM_ALLREDUCE_USE_FLASHINFER_PCIE_IPC=1","--entrypoint vllm ds41-cmp170hx:latest","serve /models/DeepSeek-V4.1-Flash-1017-hybrid","--quantization exl3 --load-format safetensors --safetensors-load-strategy lazy","--tensor-parallel-size 4 --pipeline-parallel-size 1","--served-model-name deepseek-v4.1-flash","--host 0.0.0.0 --port 8000","--max-model-len 262144 --max-num-seqs 8 --max-num-batched-tokens 2048 --gpu-memory-utilization 0.95","--kv-cache-dtype fp8_ds_mla --enable-prefix-caching","--compilation-config {\"cudagraph_mode\": \"FULL_AND_PIECEWISE\", \"cudagraph_capture_sizes\": [1, 2, 3, 4, 5, 6, 8, 12, 16, 18, 20, 24, 28, 30, 32, 36, 42, 48]}","--async-scheduling","--speculative-config {\"method\": \"dspark\", \"num_speculative_tokens\": 5}","--tokenizer-mode deepseek_v41","--enable-auto-tool-choice --tool-call-parser deepseek_v41 --reasoning-parser deepseek_v41"],"layouts":["tp","pp"],"env_defaults":{"dsv41_dynk":"DSV41_DYNK=accept:3,5 is the tp4-only default set in serve.sh:56 (per-step DSpark verify width 3 or 5 from recent acceptance; empty = always 5 drafts). pp4 sets no default (off: k = num_speculative_tokens)","dsv41_gate_tau":"DSV41_GATE_TAU is never set by serve.sh; code default 0.10 (patches/dsv41_variants/variants/dsv41_gate.py:65 DEFAULT_TAU; 0 = off); forwarded via the OPT_ENV loop serve.sh:68-74 only when set","dsv41_variants":"tp4: unset in serve.sh - falls through to the module default of TWELVE variants (patches/dsv41_variants/dsv41_variants.py:14 DEFAULT: topk_meta,h2d_lean,ragged_lean,gate,ce_ring,split_reduce,moe_try,engram_prefetch,mhc_pcfg,moe_v2try,prefill_pack,attn_slices). pp4: serve.sh:64 pins the FOUR of 1.1.0 (dsv41_topk_meta,dsv41_h2d_lean,dsv41_ragged_lean,dsv41_gate) - pipeline parallel keeps the 1.1.0 list","tp4_cudagraph_sizes":"tp4 18 sizes [1,2,3,4,5,6,8,12,16,18,20,24,28,30,32,36,42,48] (serve.sh:54; 16/20/28/32 added for decode steps that verify 3 drafts per request via DSV41_DYNK); pp4 keeps the 1.1.0 14 sizes with cudagraph_mode PIECEWISE","dsv41_draft_head_fp8":"DSV41_DRAFT_HEAD_FP8=1 in serve.sh COMMON_ENV (serve.sh:46); applies to tp4 AND pp4 (layout-independent); code default 0 (opt-in), consumed by patches/patch_v38.py:76"},"kv_pool_tokens":1395731,"max_context_len":262144,"storage_requirements":{"note":"~400 GiB fast NVMe for checkpoints (197 GB LibertAI + 18 GB official + 190 GB Engram shards) + ~40 GiB image; 64 GB+ host RAM (page cache for Engram rows).","ram_gb":64,"nvme_gb":440}},"models":[{"slug":"deepseek-v4-1-flash-1017-exl3-2-9bpw-hybrid","name":"DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw-Hybrid","hf_repo":"LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw","quant":"exl3-2-9bpw-routed-experts-fp8-core-hybrid"}],"hardware":[{"slug":"nvidia-cmp-170hxx4","vendor":"nvidia","card":"cmp-170hx","card_family":"170hx","vram_gb":64,"device_count":4}],"generation":"ampere","tags":["community-recipe","dspark","engram-nvme","exl3","pp4","sm80","speculative","tp4"],"tasks":["chat","code","vision"],"aa":{"aa_index":45,"aa_provenance":"measured","aa_source_url":"https://artificialanalysis.ai/models/deepseek-v4-1-flash-1017","aa_index_version":"Index v4.3.2 (2026-10-18)","aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-19 12:00:00-06","aa_family_slug":"deepseek-v4-1-flash-1017","aa_resolved_slug":"deepseek-v4-1-flash-1017"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"claimed_by=submitter T0 (author's bench; recipe 1.2.0, MiaAI-Lab protocol via bench/mia.py unmodified, measured 2026-10-26 on 4x CMP 170HX: 74 SMs/card, PCIe 2.0 x16, P2P on, 180 W/card, Engram on NVMe, 262,144-token context, 8 sequences, .env defaults \u2014 docs/results.md:3-5 @ 2ad5741). c1 = median of 5 streaming runs; c2/c4/c8 = median of 3 aggregate runs (results.md:11-15). 1.2.0 TP4: c1 355.7/258.4/183.6, c2 433.1/243.8/238.8, c4 626.4/387.8/348.7, c8 867.6/544.0/440.5 (results.md:19-22). 1.2.0 PP4: c1 117.5/90.5/63.2, c2 118.4/83.4/65.2, c4 205.9/142.9/105.1, c8 340.5/213.9/163.1 (results.md:19-22). Previously published 1.0.0 comparators: TP4 296/212/150 c1 and 796/477/359 c8; PP4 111/81/52 c1 and 320/188/144 c8. Cold prefill TP4 2,416 (1.1.0 1,764; +37.4 % vs the published 1,759); PP4 2,311 (1.1.0 2,457; -2.7 % vs the published 2,375, inside the +/-3-7 % boot spread; results.md:41-43, 51-52). KV pool at 262,144: TP4 1,395,731 tokens = 2.95 GiB/card, 5.3 full-length requests (results.md:44; docs/kv-and-context.md:13-14); at 1,048,576: 1,270,789 tokens = 2.27 GiB, 1.21 (results.md:45; kv-and-context.md:13). DSpark drafts 5 tokens/step; TP4 verify width DSV41_DYNK accept:3,5 (prose A/B: 2 users 232->248, 4 users 306/316->349, 8 users 420/419->445; results.md:31-34); confidence gate default tau 0.10 (prose c1 174.7 ungated vs 179.9 gated; results.md:35-37). 1.2.0 DynK/variant gains are described in results.md:31-37 prose; docs/tuning-log.md Kept table last row remains the 1.1.0 confidence gate (tuning-log.md:32); 1.2.0 tried-and-dropped incl. the 4,096-token prefill chunk (~+1 %, cold bench OOM; tuning-log.md:91) and moe_prefill_v2 (slower; tuning-log.md:95) are in the 'Tried and dropped for 1.2.0' section (tuning-log.md:86-99). Quality: HumanEval 154/164 with 6 length-capped (regression vs the published 156/164; author McNemar p >= 0.5 \u2014 run-to-run spread; results.md:60-67), GSM8K 1,275/1,319 (results.md:63); paired NLL +0.57 % (1.7 sigma) vs 1.1.0 (results.md:69-71); needle 6/6 at 19K/72K, 3/3 at 388K 1M option (results.md:80-81). Soak 20 min x 8 streams: 1,018 requests, 403,570 tokens, 0 errors, 334.8 tok/s mean (results.md:85-86). Nobody has booted 1.2.0 in this fleet \u2014 every number is the author's own measurement. Comparators: sibling EXL3/TC-MoE recipe measured 180.4/137.1/96.6 c1 and 621.1/376.1/282.5 c8 at 140 W \u2014 NOT power-matched (180 W caps + Engram-NVMe tier).\", \"tok_s\": 355.7, \"context_len\": 262144}"}],"perf_points":[{"id":1966,"context_len":10240,"prefill_tok_s":2416,"decode_tok_s":null,"ttft_ms":null,"concurrency":null,"claimed_by":"submitter","notes":"1.2.0 cold prefill median of 9 (TP4): 3 public-domain texts of ~24K/~34K/~38K tokens, 3 runs each, no cache hits \u2014 docs/results.md:43 @ 2ad5741. 1.1.0 1,764; +37.4 % vs the previously published 1,759 (1.0.0). context_len 10240 is the tuple marker; concurrency omitted (NULL). Author (T0)."},{"id":1967,"context_len":20480,"prefill_tok_s":2311,"decode_tok_s":null,"ttft_ms":null,"concurrency":null,"claimed_by":"submitter","notes":"1.2.0 cold prefill median of 9 (PP4, measured on the same x16 host) \u2014 docs/results.md:43 @ 2ad5741. 2,311 vs 1.1.0 2,457 and the previously published 2,375 (-2.7 %, inside the cold-bench boot-to-boot spread +/-3-7 %; results.md:51-52). context_len 20480 is the tuple marker; concurrency omitted (NULL). Author (T0)."},{"id":1968,"context_len":262144,"prefill_tok_s":null,"decode_tok_s":355.69999999999998863131622783839702606201171875,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"1.2.0 structured decode c1 headline (TP4, P2P on), median of 5 streaming runs \u2014 docs/results.md:19 @ 2ad5741 (1.1.0 comparator 336.0). Author measurement (claimed_by=submitter T0)."},{"id":1969,"context_len":262144,"prefill_tok_s":null,"decode_tok_s":867.6000000000000227373675443232059478759765625,"ttft_ms":null,"concurrency":8,"claimed_by":"submitter","notes":"1.2.0 aggregate structured decode c8 (TP4), median of 3 aggregate runs \u2014 docs/results.md:22 @ 2ad5741 (1.1.0 813.1). c2 433.1 / c4 626.4 in results.md:20-21. Author (T0)."},{"id":1970,"context_len":272384,"prefill_tok_s":null,"decode_tok_s":258.3999999999999772626324556767940521240234375,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"1.2.0 code decode c1 (TP4) \u2014 docs/results.md:19 @ 2ad5741 (1.1.0 246.8). context_len is the tuple marker 262144+10240 (code row, not structured). Author measurement (T0)."},{"id":1971,"context_len":272384,"prefill_tok_s":null,"decode_tok_s":544,"ttft_ms":null,"concurrency":8,"claimed_by":"submitter","notes":"1.2.0 aggregate code decode c8 (TP4) \u2014 docs/results.md:22 @ 2ad5741 (1.1.0 504.8). context_len is the tuple marker 262144+10240. c2 243.8 / c4 387.8 (results.md:20-21). Author (T0)."},{"id":1972,"context_len":282624,"prefill_tok_s":null,"decode_tok_s":183.599999999999994315658113919198513031005859375,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"1.2.0 prose decode c1 (TP4) \u2014 docs/results.md:19 @ 2ad5741 (1.1.0 180.0). context_len is the tuple marker 262144+20480 (prose row). Author measurement (T0)."},{"id":1973,"context_len":282624,"prefill_tok_s":null,"decode_tok_s":440.5,"ttft_ms":null,"concurrency":8,"claimed_by":"submitter","notes":"1.2.0 aggregate prose decode c8 (TP4) \u2014 docs/results.md:22 @ 2ad5741 (1.1.0 385.1). context_len is the tuple marker 262144+20480. c2 238.8 / c4 348.7 (results.md:20-21). Author (T0)."},{"id":1974,"context_len":303104,"prefill_tok_s":null,"decode_tok_s":117.5,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"1.2.0 structured decode c1 (PP4) \u2014 docs/results.md:19 @ 2ad5741 (1.1.0 118.3). context_len is the tuple marker 262144+40960 (PP4 row, not TP4). Author measurement (T0)."},{"id":1975,"context_len":303104,"prefill_tok_s":null,"decode_tok_s":340.5,"ttft_ms":null,"concurrency":8,"claimed_by":"submitter","notes":"1.2.0 aggregate structured decode c8 (PP4) \u2014 docs/results.md:22 @ 2ad5741 (1.1.0 340.6). context_len is the tuple marker 262144+40960; c2 118.4 / c4 205.9 (results.md:20-21). Author (T0)."},{"id":1976,"context_len":313344,"prefill_tok_s":null,"decode_tok_s":90.5,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"1.2.0 code decode c1 (PP4) \u2014 docs/results.md:19 @ 2ad5741 (1.1.0 92.2). context_len is the tuple marker 262144+51200=313344 (PP4 code row, distinct from the structured tuple). Author measurement (T0)."},{"id":1977,"context_len":313344,"prefill_tok_s":null,"decode_tok_s":213.900000000000005684341886080801486968994140625,"ttft_ms":null,"concurrency":8,"claimed_by":"submitter","notes":"1.2.0 aggregate code decode c8 (PP4) \u2014 docs/results.md:22 @ 2ad5741 (1.1.0 215.4). context_len is the tuple marker 262144+51200=313344; c2 83.4 / c4 142.9 (results.md:20-21). Author (T0)."},{"id":1978,"context_len":323584,"prefill_tok_s":null,"decode_tok_s":63.2000000000000028421709430404007434844970703125,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"1.2.0 prose decode c1 (PP4) \u2014 docs/results.md:19 @ 2ad5741 (1.1.0 63.8). context_len is the tuple marker 262144+61440=323584 (PP4 prose row). Author measurement (T0)."},{"id":1979,"context_len":323584,"prefill_tok_s":null,"decode_tok_s":163.099999999999994315658113919198513031005859375,"ttft_ms":null,"concurrency":8,"claimed_by":"submitter","notes":"1.2.0 aggregate prose decode c8 (PP4) \u2014 docs/results.md:22 @ 2ad5741 (1.1.0 160.5). context_len is the tuple marker 262144+61440=323584; c2 65.2 / c4 105.1 (results.md:20-21). Author (T0)."}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-engram-nvme-tp4-pp4-4x-cmp-170hx.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-engram-nvme-tp4-pp4-4x-cmp-170hx.json"},"requirements":{"requires":["Docker (or a compatible OCI container runtime)","NVIDIA Container Toolkit (nvidia-container-toolkit) - exposes host GPUs to the container"],"driver_floor":{"cuda":null,"linux_min":null,"text":"unknown - check the image tag","source":"NVIDIA CUDA Toolkit Release Notes - driver compatibility tables (https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/, fetched 2026-10-03)"},"placeholders":["Hardware-bound values in this recipe (device counts, bind-mount paths, utilization, ports) describe the author's machine, not yours - map them to your own rig before launching. Keep the engine flags, model, quant, and parallelism strategy; adjust paths and ports to the hardware you actually have.","GPU identity: list your GPUs with `nvidia-smi -L` and substitute your own GPU UUIDs/indices wherever the recipe pins the author's.","Model directory: bind-mount paths are author-side - point them at your own mount where the model files live.","Host port: pick a free port on your machine and publish the container there; keep the in-container port unchanged."]},"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017","pin":"official FP8 attention/shared experts/head/DSpark drafter (incl. FP4 drafter experts)/Engram shards 47-48; bf16 embeddings + LM head + vision tower"},{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw","pin":"routed experts layers 0-39 ONLY (LibertAI 2.9 bpw avg; layers 18-22 at 2 bpw); repo watched @ 0be63c3337efbb2a0061d5b8bb4006fa186d433f"},{"kind":"engine_image","pointer_type":"git_commit","pointer":"https://github.com/GNmac-lab/deepseek-v41-flash-1017-cmp170hx","pin":"repo HEAD 2ad5741ab1 (author recipe 1.2.0) at sweep 2026-10-28 (was bbaf367291 at ingest sweep 2026-10-22); image built locally on first use (~30-45 min) from pinned upstream sources via patches/ (34 patch steps)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/zebgop-ops/dsv41reap-pp","pin":"sm_80 sparse attention + PP shadow KV + DSpark V1 + Engram row store drop-ins (@ 138e7488, watched)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/vcruz305/vllm-exl3","pin":"EXL3 plugin (AGPL-3.0-only) this recipe's patches modify"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/CurtisAccelerate/DeepSeekv41-Flash-EXL3-TC-MoE","pin":"credited for the TP4 hybrid-checkpoint recipe + tensor-core EXL3 MoE port lineage (watched @ e559217ac4)"}],"links":{"self":"/api/v1/recipes/deepseek-v4-1-flash-1017-engram-nvme-tp4-pp4-4x-cmp-170hx.json","alternate":"/r/deepseek-v4-1-flash-1017-engram-nvme-tp4-pp4-4x-cmp-170hx","describedby":"/openapi.json"},"updated_at":"2026-10-28 23:26:24.539707-06","created_at":"2026-10-24 10:35:18.77262-06","speeds":{"decode_tok_s":150.599999999999994315658113919198513031005859375,"prefill_tok_s":0,"leaderboard":"/api/v1/leaderboard.json"}}