diff --git a/recipes/deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5/recipe.yaml b/recipes/deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5/recipe.yaml index 2156a097a..23f86e475 100644 --- a/recipes/deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5/recipe.yaml +++ b/recipes/deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5/recipe.yaml @@ -1,5 +1,5 @@ slug: deepseek-v4-1-flash-sm80-deepinfer-kernel-port-6x-a100-dspark-k5 -title: 'DeepSeek-V4.1-Flash SM80 DeepInfer kernel port 6x A100 DSpark K5' +title: 'DeepSeek-V4.1-Flash SM80 DeepInfer kernel port 6x CMP 170HX DSpark K5' engine: name: vllm version: 'V1 + Deepinfer-SM80 patch @ 2a828fc2122b' @@ -7,7 +7,7 @@ engine: git_url: 'https://github.com/HalfVulpes/Deepinfer-SM80' pinned_commit: 2a828fc2122b6fc59805c0c98b6aaf78e7da9b48 install_note: 'Option A: git apply patches/vllm-sm80-deepinfer.patch from a vLLM V1 checkout root (all 6 files, 1,340 lines). Option B: bind-mount/copy vllm_sm80_deepinfer/vllm/* over site-packages/vllm/ (docker bind-mount read-only overlays work - the CMP-recipe pattern). Verify with the three in-repo microbenchmarks (microbench_sm80_kernels.py / _part2 / _part3) which print bit-exactness + latency tables; charts via generate_charts.py; full dataset in benchmarks/results/sm80_benchmark_summary.json. Requirements: SM80 (A100 40/64/80 GB, A800, GA100) or SM86 (RTX 3090/3090 Ti, RTX A6000, A40/A10/A30 - all kernels <=48 KB SRAM/CTA); vLLM V1 engine; Triton + TileLang JIT (startup cache-warm included). Base checkpoint: official deepseek-ai/DeepSeek-V4.1-Flash FP8.' -summary: "DeepSeek-V4.1-Flash served on six 64 GB SM80 GPUs (validated on A100/GA100) with HalfVulpes' Deepinfer-SM80 patch set: a 6-file vLLM V1 overlay porting DeepSeek's DeepInfer kernel architecture (SM100 tcgen05/TMEM) to Ampere mma.sync tensor cores - unified 512-dim SM80 sparse-MLA decode/prefill kernels with bitshift UE8M0 scale decoding, a bit-exact single-CTA Triton TopK+SWA combiner (22.17x), direct-store KV-cache gather (7.64x @48K), SM80-aligned TileLang mHC routing (2.89x), and YOCO L20-L39 ragged indexer-metadata reuse (19 redundant kernel chains eliminated per chunk). TP2 x PP3 (14/14/12) + DSpark K=5 + 1M context, 4.65M-token GPU-resident KV. Author-measured: 64K cold prefill 4,507 tok/s (+24.2% vs baseline vLLM SM80), 64K TTFT 10.7 s (-18.9%), c1 DSpark decode 160.8 tok/s code / 188.8 tok/s 16K+ long-gen, c4 170.5 aggregate. Microbenchmarks bit-exact (0 diff) or within BF16 noise. Topology guide: PP6 = 2.1x slower single-stream; TP2+TP4 asymmetric hurts Marlin tiles." +summary: "DeepSeek-V4.1-Flash served on six 64 GB SM80 GPUs (GA100 die, owner-classified 170HX) with HalfVulpes' Deepinfer-SM80 patch set: a 6-file vLLM V1 overlay porting DeepSeek's DeepInfer kernel architecture (SM100 tcgen05/TMEM) to Ampere mma.sync tensor cores - unified 512-dim SM80 sparse-MLA decode/prefill kernels with bitshift UE8M0 scale decoding, a bit-exact single-CTA Triton TopK+SWA combiner (22.17x), direct-store KV-cache gather (7.64x @48K), SM80-aligned TileLang mHC routing (2.89x), and YOCO L20-L39 ragged indexer-metadata reuse (19 redundant kernel chains eliminated per chunk). TP2 x PP3 (14/14/12) + DSpark K=5 + 1M context, 4.65M-token GPU-resident KV. Author-measured: 64K cold prefill 4,507 tok/s (+24.2% vs baseline vLLM SM80), 64K TTFT 10.7 s (-18.9%), c1 DSpark decode 160.8 tok/s code / 188.8 tok/s 16K+ long-gen, c4 170.5 aggregate. Microbenchmarks bit-exact (0 diff) or within BF16 noise. Topology guide: PP6 = 2.1x slower single-stream; TP2+TP4 asymmetric hurts Marlin tiles." license: Apache-2.0 model: name: DeepSeek-V4.1-Flash @@ -19,10 +19,10 @@ model: hardware: - vendor: nvidia - card: A100 + card: cmp-170hx vram_gb: 64 device_count: 6 - driver: 'CUDA 13.x, validated on GA100 64GB' + driver: 'CUDA 13.x, validated on GA100 64GB (CMP 170HX)' tasks: - chat - code @@ -110,4 +110,4 @@ provenance: source_urls: - 'https://github.com/HalfVulpes/Deepinfer-SM80' - 'https://github.com/deepseek-ai/DeepInfer' - notes: "Community ingest of HalfVulpes' port (Apache-2.0). All numbers claimed_by=submitter T0 (microbench + E2E harness + raw JSON in-repo) - import via POST /api/v1/benchmarks to upgrade. HARDWARE CAVEAT: validated on GA100/A100 64 GB SM80; CMP 170HX is the same GA100 silicon at 140 W (74 SMs) but the author did not measure CMP-class power/link configs, and the 6x rig uses PCIe x16 Gen4/Gen5 - do NOT present the numbers as CMP-measured. Engine is a vLLM patch, NOT a new engine enum. Upstream deepseek-ai/DeepInfer itself stays SM100/Ascend-only and is NOT separately ingestable. Discord corroboration: cmpunlocker #testing-general / #clanker-talk 2026-10-20 17:12-17:22Z ('160.8-188.8 tok/s... even pp6 65.5-68.5... double of GLM 5.3... multi-model') and reala/segmond linking the repo (not re-verified). docs/05's KV math section names the -1017 revision while README/summary serve the base V4.1-Flash checkpoint. A 170hx-x4 CMP variant is the natural follow-up once someone measures it." + notes: "Community ingest of HalfVulpes' port (Apache-2.0). All numbers claimed_by=submitter T0 (microbench + E2E harness + raw JSON in-repo) - import via POST /api/v1/benchmarks to upgrade. HARDWARE CAVEAT: validated on GA100/A100 64 GB SM80; CMP 170HX is the same GA100 silicon at 140 W (74 SMs) but the author did not measure CMP-class power/link configs, and the 6x rig uses PCIe x16 Gen4/Gen5 - do NOT present the numbers as CMP-measured. Engine is a vLLM patch, NOT a new engine enum. Upstream deepseek-ai/DeepInfer itself stays SM100/Ascend-only and is NOT separately ingestable. Discord corroboration: cmpunlocker #testing-general / #clanker-talk 2026-10-20 17:12-17:22Z ('160.8-188.8 tok/s... even pp6 65.5-68.5... double of GLM 5.3... multi-model') and reala/segmond linking the repo (not re-verified). docs/05's KV math section names the -1017 revision while README/summary serve the base V4.1-Flash checkpoint. A 170hx-x4 CMP variant is the natural follow-up once someone measures it. HARDWARE RECLASSIFICATION (owner ruling 2026-10-21): the author labeled the rig '6x A100'; 64 GB SM80 cards are CMP 170HX (GA100 die) - real A100 is 40/80 GB and never 64 - so the recipe is owner-classified as 6x CMP 170HX; the -6x-a100- slug keeps its historical token. Author wording 'A100/GA100' is preserved in quotes inside performance notes and install_note only."