Recipes — LocalAI Recipes

Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.

DeepSeek-V4.1-Flash-1017 streaming NVMe stack on 1x RTX 5090 (fronteagle739)

by fronteagle739 - recipe, streaming stack and measurements↗, llama.cpp / ggml (MIT)↗, antirez/ds4 (MIT)↗, IngeniousIdiocy/ds4-v41-m3ultra (MIT)↗, DeepSeek - DeepSeek-V4.1-Flash-1017 base model↗ · listed by localai-recipes

Streams a 483 GiB DeepSeek-V4.1-Flash-1017 pack from NVMe straight into one 32 GB RTX 5090, so a frontier-size MoE runs on a desktop with 32 GB of VRAM and 32 GB of system RAM. Win

DeepSeek-V4.1-Flash-1017 llama.cpp NVIDIA RTX 5090 32GB ×1 (Blackwell) 17.8 tok/s decode · 3356 tok/s prefill

GLM-5.3-Flash exlr8 EXL3 experts on DGX Spark TP1-4

by kindlingai↗, nvidia↗ · listed by localai-recipes

GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3

DeepSeek-V3 TP vllm +5

Qwen3.8-Flash-Next W4A16-AutoRound DGX Spark vLLM MTP4 Iter7g

by kyaky↗ · listed by localai-recipes

Recipe by kyaky. Qwen3.8-Flash-Next W4A16-AutoRound hybrid (INT4 experts + FP8 side layers + INT8 head; 51B-token n-gram PLE table on NVMe mmap) on one DGX Spark (GB10, 121 GB unif

Qwen3.8-Flash-Next vllm NVIDIA DGX Spark 128GB ×1 (Blackwell) 45.7 tok/s decode

Qwen3.8-Flash-Next NVFP4 DGX Spark GB10 SGLang NEXTN MTP

by Michael Bemler (Bemler Labs)↗ · listed by localai-recipes

Production single-node DGX Spark (Grace Blackwell GB10, SM121, 128GB unified memory) recipe for Qwen3.8-Flash-Next on SGLang: ModelOpt NVFP4 quantization with zero-copy NVMe mmap o

Qwen3.8-Flash-Next sglang NVIDIA DGX Spark 128GB ×1 (Blackwell) 110.4 tok/s decode

Qwen3.8-Flash-Next FP4-Local 1x DGX Spark vLLM b12x

by sojufx↗ · listed by localai-recipes

Recipe by sojufx. Serves NVIDIA's nvidia/Qwen3.8-Flash-Next-FP4-Local (96 GiB NVFP4 quantization-aware-distilled checkpoint in a new compressed NVFP4 scale format) on one DGX Spark

Qwen3.8-Flash-Next vllm NVIDIA DGX Spark 128GB ×1 (Blackwell) 66.2 tok/s decode

Nemotron 3.5 Bolt 74B-A9B 1x DGX Spark vLLM b12x LiLiCorr 262K

by hasso5703↗ · listed by localai-recipes

NVIDIA Nemotron 3.5 Bolt 74B-A9B-24GB (hybrid Mamba-2/Transformer latent-MoE, 74B with 9B active, IQ2_XXS 2-bit experts, NVFP4 Mamba layers, Q8_0 rest; text+image+video in, 262,144

Nemotron-3.5-Bolt-74B-A9B-24GB vllm NVIDIA DGX Spark 128GB ×1 (Blackwell) 74.8 tok/s decode

DeepSeek-V4.1-Flash on 4x DGX Spark — tuned TP4 SGLang/DSpark

by knapcio (TP4 profile, v2.1-v2.3 production stack, release gates)↗, MiaAI-Lab (upstream: launcher, Engram store, b12x routing, memory model)↗, 0xSero (original 4x RTX PRO 6000 recipe skeleton boot.py derives from)↗, kpham-sgl (sglang#39187 bounded dense-indexer prefill transient, backported as adapter/indexer_chunked*.py)↗, BBuf + dsv4.1 branch contributors (#39370, #39646, #39648, #39653 decode kernel work in the canary images)↗, rhys101 (SG17 RoCEnante overlay + SG18 prefill TP split from DeepSeek-V4.1-Flash-vLLM-DGX-Spark-8)↗, local-inference-lab — Luke Alonso & Jason (b12x: RoCEnante one-shot RDMA all-reduce + fused MoE kernels)↗, FujitsuPolycom + sparkring contributors (switchless-ring opposite-node paths, path-aware RoCEnante)↗, Saolence (#1/#2/#4/#5/#6: ring as opt-in switch, NCCL/build fixes, first-deployer review)↗, rsync/rchmagos (#8: RoCEnante on the switchless ring, DSV41_ROCE_RING + scripts/ring_mesh/)↗ · listed by localai-recipes

DeepSeek-V4.1-Flash sglang NVIDIA DGX Spark 128GB ×4 (Blackwell) 89.7 tok/s decode · 5398 tok/s prefill

Qwen3.8-27B SGLang NVFP4 DFlash2 on 1x DGX Spark (deterministic)

by hasso5703↗ · listed by localai-recipes

First dense Qwen3.8-27B recipe on 1x DGX Spark (GB10, 128 GB unified): SGLang v0.5.22 release image + NVFP4 target + DFlash2 speculative decoding (NVFP4 calibrated 5-layer draft, d

Qwen3.8-27B sglang NVIDIA DGX Spark 128GB ×1 (Blackwell) 68.3 tok/s decode

Qwen3.8-Flash-Next NVFP4 1x DGX Spark vLLM 71.3 c1 (salient-data)

by Jeremy Newhouse / salient-data (recipe, patches, measurements)↗, tonyd2wild (bench_categories.py @ 6ad1c8f + comparator stack/numbers)↗, NVIDIA (Qwen3.8-Flash-Next-NVFP4 checkpoint; HF token gated)↗, vLLM project (nightly base image 8e2beaa3)↗, Sigstore/cosign (image signing + SBOM timestamping)↗ · listed by localai-recipes

Qwen3.8-Flash-Next (~180B multimodal MoE, NVFP4) on one NVIDIA DGX Spark (GB10, 121 GiB unified memory) with a pinned vLLM nightly image + 14 provenance-tracked patches (patch --fu

Qwen3.8-Flash-Next TP vllm +2

GLM-5.3-Flash VQ Q4_K 1x DGX Spark 1M ctx (JLFN)

by JLFN↗, Baekpica↗ · listed by localai-recipes

Recipe by JLFN (dgx-recipe, verified on-box 2026-10-17). Baekpica/GLM-5.3-Flash-VQ-GGUF (72 GiB GVQ1 release directory: routed experts vector-quantized with codebook, 316 nonexpert

GLM-5.3-Flash other NVIDIA DGX Spark 128GB ×1 (Blackwell) 24.3 tok/s decode

GLM-5.3-Flash NVFP4 4x DGX Spark switchless ring TP4 DFlash2

by Alex Ellis / OpenFaaS Ltd↗ · listed by localai-recipes

Recipe by Alex Ellis / OpenFaaS Ltd. GLM-5.3-Flash (320B-A18B, NVFP4) served tensor-parallel 4 across four DGX Spark (GB10, sm_121) nodes cabled as a closed switchless RoCE ring: t

GLM-5.3-Flash TP vllm +2

Qwen3.8-Flash-Next Uncensored NVFP4 BF16 PLE DGX Spark vLLM MTP2

by YSLAB-ai (BF16 PLE support, DGX Spark fixes, guarded MTP overlay, runtime qualification)↗, blazux (originated the FP8 PLE mmap work)↗ · listed by localai-recipes

DGX Spark recipe running NVFP4 Qwen3.8-Flash-Next with its FULL-PRECISION BF16 PLE memory-mapped from NVMe on vLLM (NOT the FP8-table variants the other Spark recipes use): the com

Qwen3.8-Flash-Next vllm NVIDIA DGX Spark 128GB ×1 (Blackwell) 44.2 tok/s decode · 2055.2 tok/s prefill

Working launch recipes for local LLM inference

Community builds

Blackwell (12)