{"meta":{"count":3,"page":1,"pages":1,"per_page":12,"applied":{"engine":null,"model":null,"quant":null,"storage":null,"hardware":["radeon-ai-pro-r9700","170hx-x3"],"generation":null,"task":null,"tier":null,"tags":[],"vram":null,"search":null,"sort":"updated","featured":false},"origin":"https://localairecipes.com","openapi":"/openapi.json","updated_at":"2026-11-01T20:25:34+00:00"},"recipes":[{"slug":"qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance","title":"Qwen3.8-27B MXFP4 DFlash2 1x Radeon AI PRO R9700 WSL2 radiance","summary":"Qwen3.8-27B on a single AMD Radeon AI PRO R9700 (RDNA4 gfx1201, 32 GB) under Windows 11 + WSL2: vLLM radiance 0.9.3 fork pinned by image digest with AMD's official Quark AWQ-MXFP4 checkpoint, DFlash2 FP8 k=7 speculative drafter (59-63% acceptance, 5.1-5.4 tok per verify), hand-written W4A8 GEMM + libr4d FP8 paged-attention/GDN kernels pinned b9e42ab + rx9 patch, WSL pinned-memory fix, explicit context sizing, vision tower skipped. Author-measured 125-134 tok/s decode, 2750-2970 tok/s prefill at 1.9k. The author's defaults moved to a three-sequence long-context shape (max-num-seqs 3, batched tokens 2560, max-model-len 220000) backed by a KV-profiles pin the author labels REPORTED rather than measured, plus an opt-in KV cache subsystem (off|ram|disk) that leaves the serve command byte-identical when off. One-command PowerShell installer, OpenAI-compatible server on :8080.","tier":"community","beta":false,"generation":"amd","decode_tok_s":130.69999999999998863131622783839702606201171875,"prefill_tok_s":2762,"updated_at":"2026-10-29 23:29:38.011666-06","commit_sha":"99b06dc5a6b5393dde004bc60139fe19f74a1aeb","content_hash":"7b9f9a8cd306be8ff298ebb68ff1164fcf07a0ef","votes":0,"credit":[{"name":"mike2153 (recipe, WSL2 installer, benchmarks)","url":"https://github.com/mike2153/mbea-qwen38-dflash"},{"name":"StillDeadcode (vllm-radiance image + libr4d kernels)","url":"https://codeberg.org/StillDeadcode/libr4d"},{"name":"ggz14 (radiance-vllm-mxfp4 fork)","url":"https://codeberg.org/ggz14/radiance-vllm-mxfp4"},{"name":"magiccodingman (radiance RDNA4 kernel lineage)"},{"name":"Inco AI / tcclaviger (DFlash2 FP8 drafter requant)","url":"https://huggingface.co/tcclaviger/Qwen3.8-27B-DFlash2-FP8"},{"name":"AMD (official Qwen3.8-27B Quark AWQ-MXFP4 checkpoint + ROCm librocdxg)","url":"https://huggingface.co/amd/Qwen3.8-27B-Quark-AWQ-MXFP4"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance.json","alternate":"/r/qwen3-8-27b-mxfp4-dflash2-1x-radeon-ai-pro-r9700-wsl2-radiance"}},{"slug":"ds4-flash-3x170hx-pp3-dspark-k5","title":"DeepSeek V4 Flash-0731 abliterated on 3x CMP 170HX \u2014 vLLM patched PP3 DSpark K5","summary":"The proven 3-card pipeline-parallel recipe on CMP 170HX (sm_80, 64GB unlocked): PP=3 with manual layer partition 15,16,12 at util 0.98, FP8 KV, block-size 256, DSpark block drafting at exactly 5 speculative tokens, async scheduling, CPU pinning (dedicated physical cores, placeholder in launch args), hard memory cap 64g with swap disallowed. Structured decode 181 tok/s at 99% acceptance. Upstream reference repo: allover326/deepseek-v4-cmp170hx @ 3dd2d88 (Aug 13 2026) \u2014 documents 15,15,13 3-card split, retracted 4-card requirement, INT4 524288 ctx w/ DSV4_LOGITS_ROW_CHUNK=64, +7% decode via c3046d1 rebase patches (full source build required). First-party 15,16,12 numbers on this page are ours.","tier":"curated","beta":false,"generation":"ampere","decode_tok_s":129.900000000000005684341886080801486968994140625,"prefill_tok_s":2350,"updated_at":"2026-10-29 21:32:23.860633-06","commit_sha":"24a9ac90156459e4efb49c951590edbea775df38","content_hash":"02d56ece2249f2b3e2b1ee2ffce158e841e20d26","votes":0,"credit":[{"name":"allover326","url":"https://github.com/allover326/deepseek-v4-cmp170hx"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/ds4-flash-3x170hx-pp3-dspark-k5.json","alternate":"/r/ds4-flash-3x170hx-pp3-dspark-k5"}},{"slug":"qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle","title":"Qwen3.8-Flash-Next W4A16 3x CMP 170HX PP3 NVMe PLE fp8 KV 1M (clee)","summary":"Working PP3 recipe for Qwen3.8-Flash-Next (Qwen4Exp 180B-A6B MoE, Intel INT4 W4A16 AutoRound) on 3x CMP 170HX (SM80, unlocked 64 GiB each): the 95.4 GiB BF16 PLE n-gram table streams live from NVMe (shard-direct preadv into a fixed GPU staging buffer, never fully loaded), fp8 e4m3 KV software-decoded on SM80, mamba SSM cache bf16, and 1,048,576-token context via YaRN x4 over the 262144-native checkpoint. KV pool 5.97M tokens (~5.7 concurrent 1M sessions). W4A16 experts run GPTQ-Marlin bit-exact; no GPU P2P on this card pairing, so PP stages through host RAM with NCCL_P2P_DISABLE=1 + --disable-custom-all-reduce. Measured cold-cache temp=0: prefill 2389 tok/s @4k rising to 4690 @1M (QSA sparse crossover past ~64k), decode 73.2 -> 59.8 tok/s @1M, TTFT 223 s @1M; exact needle recall at all depths through 1M. Fork clee/vllm-sm80 on a PINNED nightly; newer nightlies not validated.","tier":"community","beta":false,"generation":"ampere","decode_tok_s":73.2000000000000028421709430404007434844970703125,"prefill_tok_s":3563,"updated_at":"2026-10-20 15:34:25.555323-06","commit_sha":"fb3da33b3299da70eb24ce68a2c772de0dc1a356","content_hash":"0dea5f8b482a14a47e6cef76e85ad322d56f9821","votes":1,"credit":[{"name":"clee (recipe, fork patches, bench + needle harness)","url":"https://github.com/clee/qwen3.8-flash-next-3x170hx"},{"name":"clee (vllm-sm80 fork: units-fix, fp8-kv-sm80, ple-pp3, ple-nvme-staged)","url":"https://github.com/clee/vllm-sm80"},{"name":"Kai / 2Wild (Qwen38-Flash-Next-4x3090 \u2014 staged-PLE architecture inspiration)","url":"https://github.com/tonyd2wild/Qwen38-Flash-Next-4x3090"},{"name":"Intel (INT4 W4A16 AutoRound quant)","url":"https://huggingface.co/Intel/Qwen3.8-Flash-Next-W4A16-AutoRound"},{"name":"froggeric (bugfixed Qwen3.8 chat templates)","url":"https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle.json","alternate":"/r/qwen3-8-flash-next-w4a16-3x-cmp-170hx-pp3-nvme-ple-fp8-kv-1m-cle"}}]}