{"meta":{"count":2,"page":1,"pages":1,"per_page":12,"applied":{"engine":null,"model":"mimo-v2-6-flash-rl","quant":null,"storage":null,"hardware":null,"generation":null,"task":null,"tier":null,"tags":[],"vram":null,"search":null,"sort":"updated","featured":false},"origin":"https://localairecipes.com","openapi":"/openapi.json","updated_at":"2026-10-31T05:06:14+00:00"},"recipes":[{"slug":"mimo-v2-6-flash-rl-309b-moe-nvfp4-4x-cmp-170hx-tp4-dflash-1m-ctx","title":"MiMo-V2.6-Flash-RL 309B MoE NVFP4 4x CMP 170HX TP4 DFlash 1M ctx (ElectroHobo)","summary":"XiaomiMiMo's MiMo-V2.6-Flash-RL (309B MoE, 15B active, text/image/video/audio input) on four CMP 170HX cards (sm_80) at the full 1,048,576-token context: 322/275/170 tok/s one user (structured/code/prose), 754/738/556 across eight, 2,720 tok/s cold prefill, 1,615,953-token KV pool. NVFP4 weights (ModelOpt mixed precision via Marlin, bf16 activations), bf16 KV, MiMo's own DFlash drafter (7 drafts one user, 7/5/3 by request count), full-decode-step CUDA graph, one engine file bind-mounted (upstream vLLM #58235 vision-sink fix), every weight on the GPUs. Morrowmake's first listed community recipe on vllm-cmp170hx; measured 2026-10-18 under the Morrowmake bench protocol.","tier":"community","beta":false,"generation":"ampere","decode_tok_s":274.8999999999999772626324556767940521240234375,"prefill_tok_s":0,"updated_at":"2026-10-20 15:34:18.178415-06","commit_sha":"12a7bfc3d0f95506c73c0dbaf6bb13286ef57281","content_hash":"6388d1808174e278654541bfad590d93a9202607","votes":0,"credit":[{"name":"ElectroHobo (Applied Integrators)","url":"https://github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe"},{"name":"Morrowmake (vllm-cmp170hx fork + engine image + start.sh script set, MIT)","url":"https://github.com/Morrowmake/vllm-cmp170hx"},{"name":"XiaomiMiMo (MiMo-V2.6-Flash-RL base model)","url":"https://huggingface.co/XiaomiMiMo"},{"name":"ProCreations (NVFP4 checkpoint)","url":"https://huggingface.co/ProCreations/MiMo-V2.6-Flash-RL-NVFP4"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/mimo-v2-6-flash-rl-309b-moe-nvfp4-4x-cmp-170hx-tp4-dflash-1m-ctx.json","alternate":"/r/mimo-v2-6-flash-rl-309b-moe-nvfp4-4x-cmp-170hx-tp4-dflash-1m-ctx"}},{"slug":"mimo-v2-6-flash-rl-omni-on-4x-cmp-170hx-pp4-official-fp8-three-t","title":"MiMo-V2.6-Flash-RL (omni) on 4x CMP 170HX \u2014 PP4, official fp8, three-tier KV, v3.3 (kaka86mm)","summary":"Four-card Ampere recipe for XiaomiMiMo MiMo-V2.6-Flash-RL (309B-MoE/15B active, omni: text/image/video/audio in, 1M ctx) on 4x CMP 170HX (64 GB unlocked, Gen2 x4, no P2P \u2014 PP-only). Official 161 GB fp8 checkpoint on wtdcode/vllm-backport sm80: PP4 partition 12,12,12,12, fp8_e4m3 KV (15.5 GiB, 2,061,355-token pool), fp8 o_proj, Marlin W8A16 for fp8 GEMMs, spec decoding OFF (MTP verify kernel cliff), three-tier KV stack (GPU + 100 GB RAM tier via LMCache PR#4410 engine-driven transfers + disk tier). Eight patches baked into image v3.3: exact QKV sharding (0 changed elements audited), MTP loader, fp8\u2192Marlin, ViT sink fix #58235, DiffKV split-KV + spec-3D verify + prefill q-tiling (3.0x) + QK-dot split #59673. Measured by the author (2026-10-04, v3.3): decode 64.5 short / 60 @19K / 56 @67K tok/s, aggregate 483 @16 / 752 @32 streams, 200K cold prefill ~3,650 tok/s (16x via q-tiling). NVFP4 fallback kept (92\u201394 tok/s). All four modalities + tools verified; thinking-off with mm input.","tier":"community","beta":false,"generation":"ampere","decode_tok_s":60,"prefill_tok_s":3650,"updated_at":"2026-10-20 15:34:07.62236-06","commit_sha":"036b9f389cc3965f671623dda781efcd7aae63bc","content_hash":"ac7293fc06185b9b9e23401a6d805f3d10a5cfe0","votes":0,"credit":[{"name":"kaka86mm","url":"https://github.com/kaka86mm/vllm-mimo-170hx"},{"name":"wtdcode (vllm-backport)","url":"https://github.com/wtdcode/vllm-backport"},{"name":"avtc (vllm-backport PR#106)","url":"https://github.com/wtdcode/vllm-backport/pull/106"},{"name":"LMCache PR#4410","url":"https://github.com/LMCache/LMCache/pull/4410"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/mimo-v2-6-flash-rl-omni-on-4x-cmp-170hx-pp4-official-fp8-three-t.json","alternate":"/r/mimo-v2-6-flash-rl-omni-on-4x-cmp-170hx-pp4-official-fp8-three-t"}}]}