DeepSeek-V4.1-Flash-1017 streaming NVMe stack on 1x RTX 5090 (fronteagle739)

community Community submission — verify on your own hardware.

recipe by fronteagle739 - recipe, streaming stack and measurements↗, llama.cpp / ggml (MIT)↗, antirez/ds4 (MIT)↗, IngeniousIdiocy/ds4-v41-m3ultra (MIT)↗, DeepSeek - DeepSeek-V4.1-Flash-1017 base model↗ · listed by localai-recipes

revision r2 · commit 1926a9bbc1 · updated 2 h ago · license MIT + BSD-3-Clause

chat single-gpu long-context windows streaming nvme-streaming directstorage moe

Streams a 483 GiB DeepSeek-V4.1-Flash-1017 pack from NVMe straight into one 32 GB RTX 5090, so a frontier-size MoE runs on a desktop with 32 GB of VRAM and 32 GB of system RAM. Windows-only: the stack is built on DirectStorage (a Windows API) and ships as PowerShell launchers plus a Windows ds4-server.exe, so prefill cannot run on Linux - see install_note. Packed-FP4 measures 200K real-text prefill at 43.2 s (4,632 tok/s) with 7.6-7.9 tok/s max-quality decode and 22.7 tok/s max-speed; Q4_K/Q8_0 measures 3,356 tok/s prefill and 7.5 / 17.8 tok/s decode. An optional 512 GB M3 Ultra can decode while the RTX prefills. Weights are never requantized on load; a quality/speed dial trades exactness for cached-expert reuse.

Launch command

'powershell -ExecutionPolicy Bypass -File examples/single-machine/dsv41-ultra-cone.ps1 -Context 1m' '# Windows only. The launcher starts the stack service, which runs the decode engine as:' 'engines/ds4-win/ds4-server.exe --model {model} --cuda --ssd-streaming --ctx 262144 --host 127.0.0.1 --port 8089' '# OpenAI-compatible API on 127.0.0.1:8088; upstream engine on 8089' '# config.json keys: model (GGUF path), runtime_root, host_budget_gib (9.25 admits whole-prompt Engram rows), lease_minutes, gpu_idle_mib,' '#   env_overrides LLAMADECODE_APPEND_DECODE_MAX=0 / LLAMADECODE_APPEND_RESIDENT_SWEEP=1 / LLAMA_DSV41_INDEX_FP4=1' '# two-machine lane: python examples/run_preset.py --preset dsv41-ultra-cone-fp4 --machine mac|rtx'

Use this recipe with your agent

Paste this into any agent chat — it fetches and applies the recipe:

Fetch https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

Or fetch it directly:

curl -fsSL https://localairecipes.com/api/v1/recipes/deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro.json

Engine

llama.cpp · source↗ · 27e5a278↗

Install note: WINDOWS-ONLY, x64. DirectStorage (a Windows API) is the load-bearing transport, launchers are PowerShell, decode engine is engines/ds4-win/ds4-server.exe. Clone github.com/fronteagle739/streaming-llama-stack @ 27e5a278. Install DirectStorage SDK binaries separately (Microsoft redistributable terms), plus Python 3.12, VS C++ Build Tools, CMake, Ninja, CUDA 13.2, Node.js 24; DirectStorage NVMe (measured RAID-0 ~25 GB/s); 32 GB RAM. Build with build.ps1, copy config.example.json to config.json, point model at the GGUF. Model files come from the original safetensors checkpoint via scripts/convert_dsv41_gguf.py; the installer never downloads weights. Supported V4.1 layouts: Q4_K experts with Q8_0/Q4_K/Q4_0 dense (measured Q4_K/Q8_0, 483 GiB), or MXFP4 tensor-core-native packed. The two FP4 files are NOT interchangeable: the RTX file carries deepseek41.expert_layout = flashinfer-sm120-mxfp8-mxfp4-v1, the Mac file is canonical ggml-mxfp4. Linux cannot prefill.

Model

DeepSeek-V4.1-Flash-1017 (packed-fp4-tensor-core-native-q4-k-q8-0-gguf) · deepseek-ai/DeepSeek-V4.1-Flash-1017↗
AA intelligence 45 (Artificial Analysis · measured)

Hardware

NVIDIA RTX 5090 32GB ×1 (Blackwell)

Context

max context: 256K tokens

KV pool

256K tokens across concurrent requests

Performance

Speed by context

Context	Conc.	Prefill	Decode	TTFT
199,952	1	4632 tok/s	7.6 tok/s	47200 ms	reported
210,192	1	4632 tok/s	22.7 tok/s	47200 ms	reported
220,432	1	3356 tok/s	7.5 tok/s	—	reported
230,672	1	3356 tok/s	17.8 tok/s	—	reported

prefill = prompt processing · decode = generation. All decode/prefill figures are shown as reported.

reported

notes	Packed-FP4 / Q4_K-Q8_0 comparison on real text (I Promessi Sposi, 199,952 tokens, password + recall). One physical context, four measured cells. context_len is a TUPLE MARKER, not a real context length: the perf table keys one row per (context_len, concurrency), so the quant x speed-mode dimension rides on the marker exactly as atlas-rc4 does. Markers: 199952 = packed FP4 max quality; 210192 = 199952+10240 packed FP4 max speed; 220432 = 199952+20480 Q4_K/Q8_0 max quality; 230672 = 199952+30720 Q4_K/Q8_0 max speed.

notes

Packed-FP4 / Q4_K-Q8_0 comparison on real text (I Promessi Sposi, 199,952 tokens, password + recall). One physical context, four measured cells. context_len is a TUPLE MARKER, not a real context length: the perf table keys one row per (context_len, concurrency), so the quant x speed-mode dimension rides on the marker exactly as atlas-rc4 does. Markers: 199952 = packed FP4 max quality; 210192 = 199952+10240 packed FP4 max speed; 220432 = 199952+20480 Q4_K/Q8_0 max quality; 230672 = 199952+30720 Q4_K/Q8_0 max speed.

Downloads

Weights https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017
safetensors checkpoint @ 1bae0e1088993b9b82de08f59de5be5a34e42170 (~475 GiB); convert with scripts/convert_dsv41_gguf.py
Reference https://github.com/fronteagle739/streaming-llama-stack
27e5a2783148190e55c3ee031569ad67731f59de (stack, launchers, conversion script, measurements)
Reference https://github.com/ggml-org/llama.cpp
b83111815e9a79949257e9d4b087206b320a3063 (import base for the streaming prefill fork)
Reference https://github.com/antirez/ds4
a0510c2ac2bec2cbffe730fa728fb7d3880f93ec (Windows ds4 ancestry base)
Reference https://github.com/IngeniousIdiocy/ds4-v41-m3ultra
b10cb40c1786d93aa2b4c819635a7b9e84b2b635 (Mac ds4 V4.1 base)
Reference https://github.com/ddalcu/mlx-serve
6991ba3ad3d356891d43211dec941cc6f9ff6dd6 (unmodified submodule)

Revision notes

Ingested 2026-10-31 from repo HEAD 27e5a278. Windows requirement is stated deliberately: the schema has no platform field, so the constraint lives in install_note and summary. Hardware pins the EXISTING rtx-5090 profile (single card, no prior recipes) - no new profile created. All five git pins resolve on GitHub and match the repo's own NOTICE. Numbers are the author's, measured on real text; this site has not run the stack. Author caveat carried: the native safetensors conversion path has not been run end to end and qualified.

Revision history

1926a9bbc1 recipe: update deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro (r2: /performance) just now diff
4f7b8eb4eb recipe: create deepseek-v4-1-flash-1017-streaming-nvme-stack-on-1x-rtx-5090-fro just now

Fork

Export

recipe.yaml (raw)