{"meta":{"count":5,"page":1,"pages":1,"per_page":12,"applied":{"engine":null,"model":null,"quant":null,"storage":null,"hardware":["dgx-spark-x2","170hx-x3"],"generation":null,"task":null,"tier":null,"tags":[],"vram":null,"search":null,"sort":"updated","featured":false},"origin":"https://localairecipes.com","openapi":"/openapi.json","updated_at":"2026-10-31T20:15:00+00:00"},"recipes":[{"slug":"glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m","title":"GLM-5.3-Flash NVFP4 on 2x DGX Spark - Atlas RC4.2 TP2, DFlash2, 1M","summary":"GLM-5.3-Flash NVFP4 (nvidia checkpoint) served by Enntity's Atlas engine with tensor+expert parallelism across two DGX Sparks over one ConnectX-7 cable, DFlash2 speculative decoding (incoai drafter), FP8-latent KV pool with prefix caching, up to 4 concurrent requests at per-request 1M context; opinionated ./start.sh stack (image from pinned engine commit, verified NVFP4 MTP overlay, optional display-carveout / disk prefix cache / KV shard) with RigMark benchmark receipts, bounded determinism (greedy output independent of concurrency), install + converter tests, and published cross-recipe comparison (RiNGSiDE vLLM TP2).","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":56,"prefill_tok_s":0,"updated_at":"2026-10-31 12:36:48.458353-06","commit_sha":"c6672f65706eb7353091e662c24981e4b478cfa7","content_hash":"840bc43a5d941be08f419fe5a1b0e7411c3d32b5","votes":0,"credit":[{"name":"Enntity (Jason McCartney)","url":"https://github.com/Enntity"},{"name":"Reiner Schmidt (GLM-5.3-Flash Atlas port)","url":"https://github.com/Mango-kid/atlas"},{"name":"Atlas-Inf contributors (upstream base)","url":"https://github.com/Atlas-Inf/atlas"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m.json","alternate":"/r/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m"}},{"slug":"glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4","title":"GLM-5.3-Flash exlr8 EXL3 experts on DGX Spark TP1-4","summary":"GLM-5.3-Flash exlr8: MoE layers are EXL3-trellis units decoded by the custom exlr8 MoE kernel; dense linears stay FP8/NVFP4. One release carries three expert widths (k2/k3/k4 = 2/3/4 bpw), chosen at boot via VLLM_EXLR8_WIDTHS. Reference topology TP=4 on four GB10 boxes; TP=1 serves the whole model (~87 GiB) on one 128 GB DGX Spark / ASUS GX10 via tp1.yaml plus a distinct MENTAT_GROUP. TP=2 and TP=3 also measured. 2026-10-19, clocks TP-locked at 2200 MHz: decode code 128.0 tok/s, prose 72.5 at TP=4; 53.0 / 29.5 at TP=1; prefill 4,393 tok/s at 8K, 4,190 at 32K. Trade vs the shipping NVFP4 build: decode +3-8%, NLL equal or better, KV pool ~+15%, prefill 7.7% lower at Second 1x-Spark GLM-5.3-Flash lane: not the VQ Q4_K GGUF lane (/r/glm-5-3-flash-vq-q4-k-1x-dgx-spark-1m-ctx-jlfn), and not a fork of the 2x ATLAS build (/r/glm-5-3-flash-nvfp4-on-2x-dgx-spark-atlas-rc4-2-tp2-dflash2-1m) whose NVFP4 Flash base this stack overlays.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":65.2000000000000028421709430404007434844970703125,"prefill_tok_s":3438.5,"updated_at":"2026-10-31 11:51:31.886893-06","commit_sha":"e576a851c441c37ac2fb42dfa0188eef41f910e0","content_hash":"237f8f6570a37cb5ddb895edbae6d611ef23660e","votes":0,"credit":[{"name":"kindlingai","url":"https://github.com/kindlingai/glm-5.3-flash-exlr8-gx10"},{"name":"nvidia","url":"https://huggingface.co/nvidia/GLM-5.3-Flash-NVFP4"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4.json","alternate":"/r/glm-5-3-flash-exlr8-exl3-experts-on-dgx-spark-tp1-4"}},{"slug":"ds4-flash-3x170hx-pp3-dspark-k5","title":"DeepSeek V4 Flash-0731 abliterated on 3x CMP 170HX \u2014 vLLM patched PP3 DSpark K5","summary":"The proven 3-card pipeline-parallel recipe on CMP 170HX (sm_80, 64GB unlocked): PP=3 with manual layer partition 15,16,12 at util 0.98, FP8 KV, block-size 256, DSpark block drafting at exactly 5 speculative tokens, async scheduling, CPU pinning (dedicated physical cores, placeholder in launch args), hard memory cap 64g with swap disallowed. Structured decode 181 tok/s at 99% acceptance. Upstream reference repo: allover326/deepseek-v4-cmp170hx @ 3dd2d88 (Aug 13 2026) \u2014 documents 15,15,13 3-card split, retracted 4-card requirement, INT4 524288 ctx w/ DSV4_LOGITS_ROW_CHUNK=64, +7% decode via c3046d1 rebase patches (full source build required). First-party 15,16,12 numbers on this page are ours.","tier":"curated","beta":false,"generation":"ampere","decode_tok_s":129.900000000000005684341886080801486968994140625,"prefill_tok_s":2350,"updated_at":"2026-10-29 21:32:23.860633-06","commit_sha":"24a9ac90156459e4efb49c951590edbea775df38","content_hash":"02d56ece2249f2b3e2b1ee2ffce158e841e20d26","votes":0,"credit":[{"name":"allover326","url":"https://github.com/allover326/deepseek-v4-cmp170hx"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/ds4-flash-3x170hx-pp3-dspark-k5.json","alternate":"/r/ds4-flash-3x170hx-pp3-dspark-k5"}},{"slug":"qwen3-8-flash-next-uncensored-nvfp4-2x-dgx-spark-sglang-tp2-roce","title":"Qwen3.8-Flash-Next Uncensored NVFP4 2x DGX Spark SGLang TP2 ROCE","summary":"Recipe by ttlogan. Production 2-node GB10 (ASUS GX10 / DGX Spark) cluster serve of jpezzulli/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-ModelOpt-NVFP4 (OrcaRouter uncensored weights repacked to ModelOpt NVFP4 with the PLE n-gram table in FP8, ~126 GiB) on the MiaAI native vLLM stack --tp 2 2-node over ConnectX/ROCE (slug name says SGLang: historical identifier - the repo now runs vLLM at ec4d38624b), MTP speculative decoding, and a REQUIRED qwen4_exp tokenizer registry overlay (BPE tokenizer.json, NOT stock tokenizer behaviour). Podman rootless deployment with boot autostart and a deepseek/glm/qwen swap orchestrator. Measured numbers unchanged (no re-measurement in d5bd003301..ec4d38624b): 40.3 tok/s single, 117.3 tok/s agg at c=4 (the locked optimum), prefill ~2900-3000 tok/s flat; 30-min c=4 soak clean (2432 req / 0 fail / health 200). License notes: PixelML patches MIT; QSA fp8-KV overlay vendored AGPL-3.0-or-later from the single-Spark recipe.","tier":"community","beta":false,"generation":"blackwell","decode_tok_s":40.2999999999999971578290569595992565155029296875,"prefill_tok_s":2936.6999999999998181010596454143524169921875,"updated_at":"2026-10-28 16:04:52.791091-06","commit_sha":"1217f615379960f758d79b04ceb4ca2a71e8e6cc","content_hash":"db96361cfc2e1999cd1d8ce8f34678e090df5f9b","votes":0,"credit":[{"name":"ttlogan","url":"https://github.com/ttlogan/qwen38-flash-next-recipe"},{"name":"PixelML/MiaAI (SM121 QSA fallback + token-0 guard patches, MIT)"},{"name":"jpezzulli (OrcaRouter uncensored ModelOpt NVFP4 + FP8 PLE repack)","url":"https://huggingface.co/jpezzulli/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-ModelOpt-NVFP4"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-uncensored-nvfp4-2x-dgx-spark-sglang-tp2-roce.json","alternate":"/r/qwen3-8-flash-next-uncensored-nvfp4-2x-dgx-spark-sglang-tp2-roce"}},{"slug":"qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle","title":"Qwen3.8-Flash-Next W4A16 3x CMP 170HX PP3 NVMe PLE fp8 KV 1M (clee)","summary":"Working PP3 recipe for Qwen3.8-Flash-Next (Qwen4Exp 180B-A6B MoE, Intel INT4 W4A16 AutoRound) on 3x CMP 170HX (SM80, unlocked 64 GiB each): the 95.4 GiB BF16 PLE n-gram table streams live from NVMe (shard-direct preadv into a fixed GPU staging buffer, never fully loaded), fp8 e4m3 KV software-decoded on SM80, mamba SSM cache bf16, and 1,048,576-token context via YaRN x4 over the 262144-native checkpoint. KV pool 5.97M tokens (~5.7 concurrent 1M sessions). W4A16 experts run GPTQ-Marlin bit-exact; no GPU P2P on this card pairing, so PP stages through host RAM with NCCL_P2P_DISABLE=1 + --disable-custom-all-reduce. Measured cold-cache temp=0: prefill 2389 tok/s @4k rising to 4690 @1M (QSA sparse crossover past ~64k), decode 73.2 -> 59.8 tok/s @1M, TTFT 223 s @1M; exact needle recall at all depths through 1M. Fork clee/vllm-sm80 on a PINNED nightly; newer nightlies not validated.","tier":"community","beta":false,"generation":"ampere","decode_tok_s":73.2000000000000028421709430404007434844970703125,"prefill_tok_s":3563,"updated_at":"2026-10-20 15:34:25.555323-06","commit_sha":"fb3da33b3299da70eb24ce68a2c772de0dc1a356","content_hash":"0dea5f8b482a14a47e6cef76e85ad322d56f9821","votes":1,"credit":[{"name":"clee (recipe, fork patches, bench + needle harness)","url":"https://github.com/clee/qwen3.8-flash-next-3x170hx"},{"name":"clee (vllm-sm80 fork: units-fix, fp8-kv-sm80, ple-pp3, ple-nvme-staged)","url":"https://github.com/clee/vllm-sm80"},{"name":"Kai / 2Wild (Qwen38-Flash-Next-4x3090 \u2014 staged-PLE architecture inspiration)","url":"https://github.com/tonyd2wild/Qwen38-Flash-Next-4x3090"},{"name":"Intel (INT4 W4A16 AutoRound quant)","url":"https://huggingface.co/Intel/Qwen3.8-Flash-Next-W4A16-AutoRound"},{"name":"froggeric (bugfixed Qwen3.8 chat templates)","url":"https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle.json","alternate":"/r/qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle"}}]}