{"meta":{"count":1,"page":1,"pages":1,"per_page":12,"applied":{"engine":null,"model":null,"quant":null,"storage":null,"hardware":null,"generation":null,"task":"stt","tier":null,"tags":[],"vram":null,"search":null,"sort":"updated","featured":false},"origin":"https://localairecipes.com","openapi":"/openapi.json","updated_at":"2026-10-31T03:28:20+00:00"},"recipes":[{"slug":"mimo-v2-6-flash-rl-omni-on-4x-cmp-170hx-pp4-official-fp8-three-t","title":"MiMo-V2.6-Flash-RL (omni) on 4x CMP 170HX \u2014 PP4, official fp8, three-tier KV, v3.3 (kaka86mm)","summary":"Four-card Ampere recipe for XiaomiMiMo MiMo-V2.6-Flash-RL (309B-MoE/15B active, omni: text/image/video/audio in, 1M ctx) on 4x CMP 170HX (64 GB unlocked, Gen2 x4, no P2P \u2014 PP-only). Official 161 GB fp8 checkpoint on wtdcode/vllm-backport sm80: PP4 partition 12,12,12,12, fp8_e4m3 KV (15.5 GiB, 2,061,355-token pool), fp8 o_proj, Marlin W8A16 for fp8 GEMMs, spec decoding OFF (MTP verify kernel cliff), three-tier KV stack (GPU + 100 GB RAM tier via LMCache PR#4410 engine-driven transfers + disk tier). Eight patches baked into image v3.3: exact QKV sharding (0 changed elements audited), MTP loader, fp8\u2192Marlin, ViT sink fix #58235, DiffKV split-KV + spec-3D verify + prefill q-tiling (3.0x) + QK-dot split #59673. Measured by the author (2026-10-04, v3.3): decode 64.5 short / 60 @19K / 56 @67K tok/s, aggregate 483 @16 / 752 @32 streams, 200K cold prefill ~3,650 tok/s (16x via q-tiling). NVFP4 fallback kept (92\u201394 tok/s). All four modalities + tools verified; thinking-off with mm input.","tier":"community","beta":false,"generation":"ampere","decode_tok_s":60,"prefill_tok_s":3650,"updated_at":"2026-10-20 15:34:07.62236-06","commit_sha":"036b9f389cc3965f671623dda781efcd7aae63bc","content_hash":"ac7293fc06185b9b9e23401a6d805f3d10a5cfe0","votes":0,"credit":[{"name":"kaka86mm","url":"https://github.com/kaka86mm/vllm-mimo-170hx"},{"name":"wtdcode (vllm-backport)","url":"https://github.com/wtdcode/vllm-backport"},{"name":"avtc (vllm-backport PR#106)","url":"https://github.com/wtdcode/vllm-backport/pull/106"},{"name":"LMCache PR#4410","url":"https://github.com/LMCache/LMCache/pull/4410"}],"uploaded_by":"localai-recipes","measured":null,"links":{"self":"/api/v1/recipes/mimo-v2-6-flash-rl-omni-on-4x-cmp-170hx-pp4-official-fp8-three-t.json","alternate":"/r/mimo-v2-6-flash-rl-omni-on-4x-cmp-170hx-pp4-official-fp8-three-t"}}]}