{"meta":{"count":5,"page":1,"pages":1,"per_page":12,"applied":{"engine":null,"model":null,"quant":null,"storage":null,"hardware":["dgx-spark-x4","dgx-spark-x2"],"generation":null,"task":null,"tier":null,"tags":[],"vram":null,"search":null,"sort":"updated","featured":false},"origin":"https://localairecipes.com","openapi":"/openapi.json","updated_at":"2026-10-31T12:27:54+00:00"},"recipes":[{"slug":"glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m","title":"GLM-5.3-Flash NVFP4 on 2x DGX Spark - Atlas RC4.2 TP2, DFlash2, 1M","summary":"GLM-5.3-Flash NVFP4 (nvidia checkpoint) served by Enntity's Atlas engine with tensor+expert parallelism across two DGX Sparks over one ConnectX-7 cable, DFlash2 speculative decoding (incoai drafter), FP8-latent KV pool with prefix caching, up to 4 concurrent requests at per-request 1M context; opinionated ./start.sh stack (image from pinned engine commit, verified NVFP4 MTP overlay, optional display-carveout / disk prefix cache / KV shard) with RigMark benchmark receipts, bounded determinism (greedy output independent of concurrency), install + converter tests, and published cross-recipe comparison (RiNGSiDE vLLM TP2).","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":0,"prefill_tok_s":0,"updated_at":"2026-10-29 21:32:26.380514-06","commit_sha":"653270250a28129f993c556f74d24012fe5ee558","content_hash":"840bc43a5d941be08f419fe5a1b0e7411c3d32b5","votes":0,"credit":[{"name":"Enntity (Jason McCartney)","url":"https://github.com/Enntity"},{"name":"Reiner Schmidt (GLM-5.3-Flash Atlas port)","url":"https://github.com/Mango-kid/atlas"},{"name":"Atlas-Inf contributors (upstream base)","url":"https://github.com/Atlas-Inf/atlas"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m.json","alternate":"/r/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m"}},{"slug":"glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4","title":"GLM-5.3-Flash exlr8 EXL3 experts on DGX Spark TP1-4","summary":"GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3/4 bpw), chosen at boot via VLLM_EXLR8_WIDTHS. Reference topology TP=4 on four GB10 boxes; TP=1 serves the whole model (~87 GiB) on one 128 GB DGX Spark / ASUS GX10 via tp1.yaml plus a distinct MENTAT_GROUP. TP=2 and TP=3 also measured. 2026-10-19, clocks TP-locked at 2200 MHz: decode code 128.0 tok/s, prose 72.5 at TP=4; 53.0 / 29.5 at TP=1; prefill 4,393 tok/s at 8K, 4,190 at 32K. Trade vs the shipping NVFP4 build: decode +3-8%, NLL equal or better, KV pool ~+15%, prefill 7.7% lower at Second 1x-Spark GLM-5.3-Flash lane: not the VQ Q4_K GGUF lane (/r/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn), and not a fork of the 2x ATLAS build (/r/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m) whose NVFP4 Flash base this stack overlays.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":65.2000000000000028421709430404007434844970703125,"prefill_tok_s":3438.5,"updated_at":"2026-10-29 21:32:25.385656-06","commit_sha":"2ddd4d836d8efa003f7203ba30597c20dc21e495","content_hash":"79db03010b10026e9788b7c749df25daf5fb38be","votes":0,"credit":[{"name":"kindlingai","url":"https://github.com/kindlingai/glm-5.3-flash-exlr8-gx10"},{"name":"nvidia","url":"https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4.json","alternate":"/r/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4"}},{"slug":"deepseek-v4-1-flash-on-4x-dgx-spark-tuned-tp4-sglang-dspark","title":"DeepSeek-V4.1-Flash on 4x DGX Spark \u2014 tuned TP4 SGLang/DSpark","summary":"","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":89.7000000000000028421709430404007434844970703125,"prefill_tok_s":5650,"updated_at":"2026-10-29 21:32:23.622356-06","commit_sha":"ddf188add74be5de70f13f6f71735ec18a87c83c","content_hash":"647b167cbdd40dfd95cbaabc81e8624e929d847d","votes":0,"credit":[{"name":"knapcio (TP4 profile, v2.1-v2.3 production stack, release gates)","url":"https://github.com/knapcio/DeepSeek-V4.1-Flash-4x-DGX-Spark-TP4"},{"name":"MiaAI-Lab (upstream: launcher, Engram store, b12x routing, memory model)","url":"https://github.com/MiaAI-Lab/DeepSeek-v4.1-Flash-DGX-Sparks"},{"name":"0xSero (original 4x RTX PRO 6000 recipe skeleton boot.py derives from)","url":"https://github.com/sybil-solutions/deepseek-v4.1-flash-4x-rtx-pro-6000"},{"name":"kpham-sgl (sglang#39187 bounded dense-indexer prefill transient, backported as adapter/indexer_chunked*.py)","url":"https://github.com/sgl-project/sglang/pull/39187"},{"name":"BBuf + dsv4.1 branch contributors (#39370, #39646, #39648, #39653 decode kernel work in the canary images)","url":"https://github.com/sgl-project/sglang"},{"name":"rhys101 (SG17 RoCEnante overlay + SG18 prefill TP split from DeepSeek-V4.1-Flash-vLLM-DGX-Spark-8)","url":"https://github.com/rhys101/DeepSeek-V4.1-Flash-vLLM-DGX-Spark-8"},{"name":"local-inference-lab \u2014 Luke Alonso & Jason (b12x: RoCEnante one-shot RDMA all-reduce + fused MoE kernels)","url":"https://github.com/local-inference-lab/b12x"},{"name":"FujitsuPolycom + sparkring contributors (switchless-ring opposite-node paths, path-aware RoCEnante)","url":"https://github.com/FujitsuPolycom/sparkring"},{"name":"Saolence (#1/#2/#4/#5/#6: ring as opt-in switch, NCCL/build fixes, first-deployer review)","url":"https://github.com/knapcio/DeepSeek-V4.1-Flash-4x-DGX-Spark-TP4/pull/4"},{"name":"rsync/rchmagos (#8: RoCEnante on the switchless ring, DSV41_ROCE_RING + scripts/ring_mesh/)","url":"https://github.com/knapcio/DeepSeek-V4.1-Flash-4x-DGX-Spark-TP4/pull/8"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/deepseek-v4-1-flash-on-4x-dgx-spark-tuned-tp4-sglang-dspark.json","alternate":"/r/deepseek-v4-1-flash-on-4x-dgx-spark-tuned-tp4-sglang-dspark"}},{"slug":"qwen3-8-flash-next-uncensored-nvfp4-2x-dgx-spark-sglang-tp2-roce","title":"Qwen3.8-Flash-Next Uncensored NVFP4 2x DGX Spark SGLang TP2 ROCE","summary":"Recipe by ttlogan. Production 2-node GB10 (ASUS GX10 / DGX Spark) cluster serve of jpezzulli/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-ModelOpt-NVFP4 (OrcaRouter uncensored weights repacked to ModelOpt NVFP4 with the PLE n-gram table in FP8, ~126 GiB) on the MiaAI native vLLM stack --tp 2 2-node over ConnectX/ROCE (slug name says SGLang: historical identifier - the repo now runs vLLM at ec4d38624b), MTP speculative decoding, and a REQUIRED qwen4_exp tokenizer registry overlay (BPE tokenizer.json, NOT stock tokenizer behaviour). Podman rootless deployment with boot autostart and a deepseek/glm/qwen swap orchestrator. Measured numbers unchanged (no re-measurement in d5bd003301..ec4d38624b): 40.3 tok/s single, 117.3 tok/s agg at c=4 (the locked optimum), prefill ~2900-3000 tok/s flat; 30-min c=4 soak clean (2432 req / 0 fail / health 200). License notes: PixelML patches MIT; QSA fp8-KV overlay vendored AGPL-3.0-or-later from the single-Spark recipe.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":40.2999999999999971578290569595992565155029296875,"prefill_tok_s":2936.6999999999998181010596454143524169921875,"updated_at":"2026-10-28 16:04:52.791091-06","commit_sha":"1217f615379960f758d79b04ceb4ca2a71e8e6cc","content_hash":"db96361cfc2e1999cd1d8ce8f34678e090df5f9b","votes":0,"credit":[{"name":"ttlogan","url":"https://github.com/ttlogan/qwen38-flash-next-recipe"},{"name":"PixelML/MiaAI (SM121 QSA fallback + token-0 guard patches, MIT)"},{"name":"jpezzulli (OrcaRouter uncensored ModelOpt NVFP4 + FP8 PLE repack)","url":"https://huggingface.co/jpezzulli/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-ModelOpt-NVFP4"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-2x-dgx-spark-sglang-tp2-roce.json","alternate":"/r/qwen3-8-flash-next-uncensored-nvfp4-2x-dgx-spark-sglang-tp2-roce"}},{"slug":"glm-5-3-flash-nvfp4-4x-dgx-spark-switchless-ring-tp4-dflash2","title":"GLM-5.3-Flash NVFP4 4x DGX Spark switchless ring TP4 DFlash2","summary":"Recipe by Alex Ellis / OpenFaaS Ltd. GLM-5.3-Flash (320B-A18B, NVFP4) served tensor-parallel 4 across four DGX Spark (GB10, sm_121) nodes cabled as a closed switchless RoCE ring: two rails per node wired to two neighbours, the non-adjacent hop relayed through a neighbour \u2014 no switch on the data path or the procurement path. Collectives use canonical switchless-nccl v0.0.1 (NCCL 2.30.7 + two Apache-2.0 switchless patches + OpenFaaS hardening patch), LD_PRELOADed with pinned library sha256 and NCCL_ALGO=Ring; not SparkRing/SIRCL, distinct from the site's switched 4x Spark SGLang lane. vLLM on the public tonyd2wild sm121-v11-dflash2 image: TP4, marlin MoE, FP8 E4M3 KV at 12 GiB per rank (pool 1,576,246 tokens = 6.01x window), DFlash2 drafter k=7, glm47 tool + glm45 reasoning parsers, max-num-seqs 6, workers (ranks 3,2,1) headless first, then head rank 0 opens the single OpenAI-compatible endpoint on :8000. Same fabric also served GLM-5.2 (EXL3, QuantTrio) and DeepSeek-V4-Flash at TP4.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":45,"prefill_tok_s":1971.59999999999990905052982270717620849609375,"updated_at":"2026-10-20 21:52:36.421715-06","commit_sha":"3ec203d32cde9b249d3e88f040a8d0c79650c3db","content_hash":"9cf1d9bb8001b5e721793f15e201aee0b5916f6d","votes":0,"credit":[{"name":"Alex Ellis / OpenFaaS Ltd","url":"https://github.com/alexellis/glm-5.3-flash-4x-dgx-spark-switchless"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/glm-5-3-flash-nvfp4-4x-dgx-spark-switchless-ring-tp4-dflash2.json","alternate":"/r/glm-5-3-flash-nvfp4-4x-dgx-spark-switchless-ring-tp4-dflash2"}}]}