{"slug":"deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro","title":"DeepSeek-V4.1-Flash-1017 streaming NVMe stack on 1x RTX 5090 (fronteagle739)","summary":"Streams a 483 GiB DeepSeek-V4.1-Flash-1017 pack from NVMe straight into one 32 GB RTX 5090, so a frontier-size MoE runs on a desktop with 32 GB of VRAM and 32 GB of system RAM. Windows-only: the stack is built on DirectStorage (a Windows API) and ships as PowerShell launchers plus a Windows ds4-server.exe, so prefill cannot run on Linux - see install_note. Packed-FP4 measures 200K real-text prefill at 43.2 s (4,632 tok/s) with 7.6-7.9 tok/s max-quality decode and 22.7 tok/s max-speed; Q4_K/Q8_0 measures 3,356 tok/s prefill and 7.5 / 17.8 tok/s decode. An optional 512 GB M3 Ultra can decode while the RTX prefills. Weights are never requantized on load; a quality/speed dial trades exactness for cached-expert reuse.","license":"MIT + BSD-3-Clause","tier":"community","beta":false,"revision":{"no":2,"commit_sha":"1926a9bbc1be021dbef26cb9199faf4531de2b9f","content_hash":"f40ac13b6d6e0cf49620410496b08fb7ce1ae6f6","created_at":"2026-10-31 12:32:41.796895-06","notes":"Ingested 2026-10-31 from repo HEAD 27e5a278. Windows requirement is stated deliberately: the schema has no platform field, so the constraint lives in install_note and summary. Hardware pins the EXISTING rtx-5090 profile (single card, no prior recipes) - no new profile created. All five git pins resolve on GitHub and match the repo's own NOTICE. Numbers are the author's, measured on real text; this site has not run the stack. Author caveat carried: the native safetensors conversion path has not been run end to end and qualified."},"engine_source":{"type":"fork","git_url":"https://github.com/fronteagle739/streaming-llama-stack","base_engine":"llama.cpp","pip_package":null,"base_version":"llama.cpp fork b83111815e9a + Windows ds4","image_digest":null,"install_note":"WINDOWS-ONLY, x64. DirectStorage (a Windows API) is the load-bearing transport, launchers are PowerShell, decode engine is engines/ds4-win/ds4-server.exe. Clone github.com/fronteagle739/streaming-llama-stack @ 27e5a278. Install DirectStorage SDK binaries separately (Microsoft redistributable terms), plus Python 3.12, VS C++ Build Tools, CMake, Ninja, CUDA 13.2, Node.js 24; DirectStorage NVMe (measured RAID-0 ~25 GB/s); 32 GB RAM. Build with build.ps1, copy config.example.json to config.json, point model at the GGUF. Model files come from the original safetensors checkpoint via scripts/convert_dsv41_gguf.py; the installer never downloads weights. Supported V4.1 layouts: Q4_K experts with Q8_0/Q4_K/Q4_0 dense (measured Q4_K/Q8_0, 483 GiB), or MXFP4 tensor-core-native packed. The two FP4 files are NOT interchangeable: the RTX file carries deepseek41.expert_layout = flashinfer-sm120-mxfp8-mxfp4-v1, the Mac file is canonical ggml-mxfp4. Linux cannot prefill.","pinned_commit":"27e5a2783148190e55c3ee031569ad67731f59de"},"credit":[{"name":"fronteagle739 - recipe, streaming stack and measurements","url":"https://github.com/fronteagle739/streaming-llama-stack"},{"name":"llama.cpp / ggml (MIT)","url":"https://github.com/ggml-org/llama.cpp"},{"name":"antirez/ds4 (MIT)","url":"https://github.com/antirez/ds4"},{"name":"IngeniousIdiocy/ds4-v41-m3ultra (MIT)","url":"https://github.com/IngeniousIdiocy/ds4-v41-m3ultra"},{"name":"DeepSeek - DeepSeek-V4.1-Flash-1017 base model","url":"https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017"}],"author":{"name":"localai-recipes","isSite":false},"uploaded_by":"localai-recipes","config":{"launch":["powershell -ExecutionPolicy Bypass -File examples/single-machine/dsv41-ultra-cone.ps1 -Context 1m","# Windows only. The launcher starts the stack service, which runs the decode engine as:","engines/ds4-win/ds4-server.exe --model {model} --cuda --ssd-streaming --ctx 262144 --host 127.0.0.1 --port 8089","# OpenAI-compatible API on 127.0.0.1:8088; upstream engine on 8089","# config.json keys: model (GGUF path), runtime_root, host_budget_gib (9.25 admits whole-prompt Engram rows), lease_minutes, gpu_idle_mib,","#   env_overrides LLAMADECODE_APPEND_DECODE_MAX=0 / LLAMADECODE_APPEND_RESIDENT_SWEEP=1 / LLAMA_DSV41_INDEX_FP4=1","# two-machine lane: python examples/run_preset.py --preset dsv41-ultra-cone-fp4 --machine mac|rtx"],"kv_pool_tokens":262144,"max_context_len":262144,"storage_requirements":{"note":"483 GiB model pack on DirectStorage NVMe at ~25 GB/s","ram_gb":32,"nvme_gb":500}},"models":[{"slug":"deepseek-v4-1-flash-1017","name":"DeepSeek-V4.1-Flash-1017","hf_repo":"deepseek-ai/DeepSeek-V4.1-Flash-1017","quant":"packed-fp4-tensor-core-native-q4-k-q8-0-gguf"}],"hardware":[{"slug":"rtx-5090","vendor":"nvidia","card":"RTX 5090","card_family":"rtx-5090","vram_gb":32,"device_count":1}],"generation":"blackwell","tags":["directstorage","long-context","moe","nvme-streaming","single-gpu","streaming","windows"],"tasks":["chat"],"aa":{"aa_index":45,"aa_provenance":"measured","aa_source_url":"https://artificialanalysis.ai/models/deepseek-v4-1-flash-1017","aa_index_version":"Index v4.3.2 (2026-10-18)","aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-19 12:00:00-06","aa_family_slug":"deepseek-v4-1-flash-1017","aa_resolved_slug":"deepseek-v4-1-flash-1017"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Packed-FP4 / Q4_K-Q8_0 comparison on real text (I Promessi Sposi, 199,952 tokens, password + recall). One physical context, four measured cells. context_len is a TUPLE MARKER, not a real context length: the perf table keys one row per (context_len, concurrency), so the quant x speed-mode dimension rides on the marker exactly as atlas-rc4 does. Markers: 199952 = packed FP4 max quality; 210192 = 199952+10240 packed FP4 max speed; 220432 = 199952+20480 Q4_K/Q8_0 max quality; 230672 = 199952+30720 Q4_K/Q8_0 max speed.\", \"tok_s\": 22.699999999999999289457264239899814128875732421875, \"ttft_ms\": 47200, \"context_len\": 199952}"}],"perf_points":[{"id":2322,"context_len":199952,"prefill_tok_s":4632,"decode_tok_s":7.5999999999999996447286321199499070644378662109375,"ttft_ms":47200,"concurrency":1,"claimed_by":"submitter","notes":"packed FP4, max quality \u2014 marker = real context 199952"},{"id":2323,"context_len":210192,"prefill_tok_s":4632,"decode_tok_s":22.699999999999999289457264239899814128875732421875,"ttft_ms":47200,"concurrency":1,"claimed_by":"submitter","notes":"packed FP4, max speed \u2014 tuple marker 199952+10240 (quant/mode dimension; NOT a real context length)"},{"id":2324,"context_len":220432,"prefill_tok_s":3356,"decode_tok_s":7.5,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"Q4_K/Q8_0, max quality \u2014 tuple marker 199952+20480 (quant/mode dimension; NOT a real context length)"},{"id":2325,"context_len":230672,"prefill_tok_s":3356,"decode_tok_s":17.800000000000000710542735760100185871124267578125,"ttft_ms":null,"concurrency":1,"claimed_by":"submitter","notes":"Q4_K/Q8_0, max speed \u2014 tuple marker 199952+30720 (quant/mode dimension; NOT a real context length)"}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro.json"},"requirements":null,"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017","pin":"safetensors checkpoint @ 1bae0e1088993b9b82de08f59de5be5a34e42170 (~475 GiB); convert with scripts/convert_dsv41_gguf.py"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/fronteagle739/streaming-llama-stack","pin":"27e5a2783148190e55c3ee031569ad67731f59de (stack, launchers, conversion script, measurements)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/ggml-org/llama.cpp","pin":"b83111815e9a79949257e9d4b087206b320a3063 (import base for the streaming prefill fork)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/antirez/ds4","pin":"a0510c2ac2bec2cbffe730fa728fb7d3880f93ec (Windows ds4 ancestry base)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/IngeniousIdiocy/ds4-v41-m3ultra","pin":"b10cb40c1786d93aa2b4c819635a7b9e84b2b635 (Mac ds4 V4.1 base)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/ddalcu/mlx-serve","pin":"6991ba3ad3d356891d43211dec941cc6f9ff6dd6 (unmodified submodule)"}],"links":{"self":"/api/v1/recipes/deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro.json","alternate":"/r/deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro","describedby":"/openapi.json"},"updated_at":"2026-10-31 12:32:41.796895-06","created_at":"2026-10-31 12:14:15.379601-06","speeds":{"decode_tok_s":12.699999999999999289457264239899814128875732421875,"prefill_tok_s":3994,"leaderboard":"/api/v1/leaderboard.json"}}