diff --git a/recipes/deepseek-v4-1-flash-1017-dsx-2-6-bit-gptq-2x-rtx-3090-engram/recipe.yaml b/recipes/deepseek-v4-1-flash-1017-dsx-2-6-bit-gptq-2x-rtx-3090-engram/recipe.yaml index c1b9e813e..57641e58a 100644 --- a/recipes/deepseek-v4-1-flash-1017-dsx-2-6-bit-gptq-2x-rtx-3090-engram/recipe.yaml +++ b/recipes/deepseek-v4-1-flash-1017-dsx-2-6-bit-gptq-2x-rtx-3090-engram/recipe.yaml @@ -2,10 +2,10 @@ slug: deepseek-v4-1-flash-1017-dsx-2-6-bit-gptq-2x-rtx-3090-engram title: 'DeepSeek-V4.1-Flash-1017 dsx 2.6-bit GPTQ 2x RTX 3090 (Engram)' engine: name: dsx - version: 'dsx 0.1.0 @ 971c072 (HEAD, unpinned upstream)' + version: 'dsx @ 98f1259732' source_type: release git_url: 'https://github.com/lxe/dsx' - pinned_commit: 971c0725bfd3906ffd292bb492b14a54ad20737f + pinned_commit: 98f1259732f8c3a5f445b25e080daf70fc300967 install_note: 'Source build: clone github.com/lxe/dsx, CUDA_ARCHS=86 cargo build --release -p dsx --features cuda (no release tags, no published image; MIT). Weights: hf download icecubetr/DeepSeek-V4.1-Flash-1017-dsx-2.6bit (187.6 GB single GGUF + vision.safetensors + expert-rank.json). Engram+tokenizer from deepseek-ai/DeepSeek-V4.1-Flash-1017: --include model.safetensors.index.json model-00047-of-00048.safetensors model-00048-of-00048.safetensors tokenizer.json. Needs 2x 24 GB (sm_86; sm_89/sm_120 build too), ~140 GB VRAM+RAM, x86 AVX2 (VNNI preferred), ~350 GB disk. OpenAI /v1 + Anthropic /v1/messages + web UI on :8090.' summary: "DeepSeek-V4.1-Flash-1017 (552B MoE + 196B Engram) quantized to 187.6 GB single GGUF (2.6127 bpw routed-expert average: 5717 PTQ1_0 / 2545 Q2_0 / 7098 Q3_K per-expert GPTQ with allocator-spent bit budget; dense Q8_0 attention/shared expert/head, F16 embedding/compressor/indexer/Engram-wkv, F32 norms/router/mHC), served by dsx, a Rust+CUDA engine built for this checkpoint, on two RTX 3090s with 192 GB RAM. ~26 tok/s decode / ~170 tok/s prefill at --ctx 262144 with all 15,360 routed experts resident. Exact routing: no expert pruning, no router bias - the rank file orders placement only. Engram n-gram tables (~203 GB) are NOT quantized: read at runtime from the official safetensor shards by a prefetch thread. Hottest ~3,300 experts in VRAM across both GPUs (GPU0's set adapts to the conversation, DSX_ADAPT=8 swaps/token), remaining experts computed on 14 CPU threads from a hugepage RAM arena. Quality vs FP8/FP4 teacher (dsx eval, Lucebox 1017 set): KL 0.1972, top-1 82.59%, PPL 9.3626." license: MIT @@ -80,7 +80,7 @@ artifacts: kind: engine_image pointer_type: git_commit pointer: 'https://github.com/lxe/dsx' - pin: '971c0725bfd3906ffd292bb492b14a54ad20737f (HEAD at ingest 2026-10-20, no release tags; MIT). NVIDIA runner in nvidia/; AMD MI325X runner (amd/) is a separate future card.' + pin: '98f1259732f8c3a5f445b25e080daf70fc300967 (HEAD at revision 2026-10-28, audited; no release tags; MIT). NVIDIA runner in nvidia/; AMD MI325X runner (amd/) is a separate future card.' provenance: credit: -