MiMo-V2.6-Flash-RL 309B MoE NVFP4 4x CMP 170HX TP4 DFlash 1M ctx (ElectroHobo)

community Community submission — verify on your own hardware.

recipe by ElectroHobo (Applied Integrators)↗, Morrowmake (vllm-cmp170hx fork + engine image + start.sh script set, MIT)↗, XiaomiMiMo (MiMo-V2.6-Flash-RL base model)↗, ProCreations (NVFP4 checkpoint)↗ · listed by localai-recipes

revision r2 · commit 12a7bfc3d0 · updated 10 d ago · license MIT

chat code vision speculative tp4 mtp nvfp4 community-recipe dflash morrowmake-fork omni

XiaomiMiMo's MiMo-V2.6-Flash-RL (309B MoE, 15B active, text/image/video/audio input) on four CMP 170HX cards (sm_80) at the full 1,048,576-token context: 322/275/170 tok/s one user (structured/code/prose), 754/738/556 across eight, 2,720 tok/s cold prefill, 1,615,953-token KV pool. NVFP4 weights (ModelOpt mixed precision via Marlin, bf16 activations), bf16 KV, MiMo's own DFlash drafter (7 drafts one user, 7/5/3 by request count), full-decode-step CUDA graph, one engine file bind-mounted (upstream vLLM #58235 vision-sink fix), every weight on the GPUs. Morrowmake's first listed community recipe on vllm-cmp170hx; measured 2026-10-18 under the Morrowmake bench protocol.

Launch command

'git clone https://github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe && cd mimo-v26-flash-cmp170hx-recipe' ./start.sh

runtime

env: "Defaults in .env (from .env.example): CUDA_VISIBLE_DEVICES=0,1,2,3 | HOST=127.0.0.1 | PORT=8000 | SERVED_MODEL_NAME=mimo-v2.6-flash | SPEC_MODE=dflash | SPEC_N=7 (dflash) / 2 (mtp) | SPEC_SCHEDULE=7/5/3 by request count (custom ranges e.g. [[1,2,7],[3,32,4]]; none = fixed SPEC_N) | SPEC_ADAPTIVE_K optional load/acceptance-adaptive depth (helps prose, costs structured) | MAX_LEN=1048576 | KV_MEM=10737418240 (10 GiB KV bytes/card ~= 1.62M tokens) | KV_DTYPE=bfloat16 (fp8_e4m3 only with mtp/none - DFlash's Triton attention has no fp8 KV on sm_80) | MAX_SEQS=32 | MAX_BATCHED=2048 (1024: -10% prefill +1.7% pool; 4096: +5% -3%) | FAIR_CHUNK=384 fair prefill slices while anyone is decoding (FAIR_PARTIAL=2) | P2P=auto (content-checked; off = ~40% slower) | EXPERT_PARALLEL=0 (measured 4-9% slower when 1) | KV_OFFLOAD_GB=0 (host-RAM KV tier; KV_OFFLOAD_DIR adds a write-through disk tier behind it) | GPU_UTIL fixed 0.94 and TP4 layout fixed (serve.sh refuses others) | FIRST_LIGHT=1 = 32K/eager/no-spec diagnostic."
notes: 'API on 127.0.0.1:8000, no key by default; reasoning in message.reasoning, answer in content; thinking off per request via chat_template_kwargs.enable_thinking=false. miMoV2OmniForCausalLM selected via --hf-overrides so checkpoint files are used as downloaded (block size 128, reasoning/tool parsers mimo, prefix caching + async scheduling on). Depth-by-request schedule gave +11/+12/+39% at eight users vs fixed 7. MTP alternative is slower on every column measured (128.9/119.6/114.5 tok/s c1). Not yet measured upstream: video/audio input, sustained load, 16 users, long-context retrieval (docs/known-limits.md).'

Use this recipe with your agent

Paste this into any agent chat — it fetches and applies the recipe:

Fetch https://localairecipes.com/api/v1/recipes/mimo-v2-6-flash-rl-309b-moe-nvfp4-4x-cmp-170hx-tp4-dflash-1m-ctx.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

Or fetch it directly:

curl -fsSL https://localairecipes.com/api/v1/recipes/mimo-v2-6-flash-rl-309b-moe-nvfp4-4x-cmp-170hx-tp4-dflash-1m-ctx.json

Engine

vllm · source↗ · 18305948↗ · ghcr.io/morrowmake/vllm-cmp170hx@sha256:5b45c4d6feafefe2eba962375b0a9a0b64fd22b9e05abb4d1966f14a32a2db79↗

Install note: One command from a fresh clone after one-time host setup: git clone https://github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe && cd mimo-v26-flash-cmp170hx-recipe && ./start.sh (preflight GPU/link/disk/port checks -> engine image pull -> checkpoint download at pinned HF revision -> detached launch -> /health wait -> two-request boot check). docker compose up is the equivalent. First start compiles kernels (later starts reuse ./cache). One engine file bind-mounted: patches/mimo_v2_omni.py with upstream vLLM #58235's vision fix (VISION_PATCH=0 disables; text unchanged, images misread). Host: 4x CMP 170HX 64 GiB on PCIe x16 links (x16-capacitor-mod cards) behind one PLX switch, P2P on (P2P=auto content-checks each start), driver >=580 (measured 610.57.04), ~190 GiB disk.

Model

MiMo-V2.6-Flash-RL (nvfp4) · XiaomiMiMo/MiMo-V2.6-Flash-RL↗
AA intelligence 38

Hardware

NVIDIA CMP 170HX 64GB ×4 (Ampere)

Context

max context: 1M tokens

KV pool

1,615,953 tokens across concurrent requests

Performance

Speed by context

Context	Conc.	Prefill	Decode	TTFT
1	—	2722 tok/s	—	—	reported
1,048,576	1	—	322 tok/s	—	reported
1,048,576	8	—	753.8 tok/s	—	reported
1,058,816	1	—	274.9 tok/s	—	reported
1,058,816	8	—	738 tok/s	—	reported
1,069,056	1	—	170.1 tok/s	—	reported
1,069,056	8	—	555.5 tok/s	—	reported

prefill = prompt processing · decode = generation. All decode/prefill figures are shown as reported.

reported

notes	claimed_by=submitter T0 (author's bench/cmp_recipe_bench.py, Morrowmake protocol: temp 0, thinking off, 400-token pin, per-request salt, median of 5 single-user / median of 3 aggregate; measured 2026-10-18, 74 SMs/card, P2P on, 180 W/card, PCIe x16). All decode points ran at the full 1M context; context_len on code/prose rows is the tuple marker 1048576+10240 (code) / +20480 (prose) to keep (context_len, concurrency) unique. Quality: HumanEval 161/164, GSM8K 195/200 (first 200, thinking on); 1M ctx accepted; image input correct with the #58235 patch. Decode step 15 ms c1 / 21 ms c2 / ~48 ms c8.

notes

claimed_by=submitter T0 (author's bench/cmp_recipe_bench.py, Morrowmake protocol: temp 0, thinking off, 400-token pin, per-request salt, median of 5 single-user / median of 3 aggregate; measured 2026-10-18, 74 SMs/card, P2P on, 180 W/card, PCIe x16). All decode points ran at the full 1M context; context_len on code/prose rows is the tuple marker 1048576+10240 (code) / +20480 (prose) to keep (context_len, concurrency) unique. Quality: HumanEval 161/164, GSM8K 195/200 (first 200, thinking on); 1M ctx accepted; image input correct with the #58235 patch. Decode step 15 ms c1 / 21 ms c2 / ~48 ms c8.

Install requirements

Docker (or a compatible OCI container runtime)
NVIDIA Container Toolkit (nvidia-container-toolkit) - exposes host GPUs to the container

NVIDIA driver floor: unknown - check the image tag

Source: NVIDIA CUDA Toolkit Release Notes - driver compatibility tables (https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/, fetched 2026-10-03).

Map the placeholders to your rig

Hardware-bound values in this recipe (device counts, bind-mount paths, utilization, ports) describe the author's machine, not yours - map them to your own rig before launching. Keep the engine flags, model, quant, and parallelism strategy; adjust paths and ports to the hardware you actually have.
GPU identity: list your GPUs with `nvidia-smi -L` and substitute your own GPU UUIDs/indices wherever the recipe pins the author's.
Model directory: bind-mount paths are author-side - point them at your own mount where the model files live.
Host port: pick a free port on your machine and publish the container there; keep the in-container port unchanged.

Downloads

Weights https://huggingface.co/ProCreations/MiMo-V2.6-Flash-RL-NVFP4
revision 34cebc5d75743a93ab76145ec609cfef9da8c417 - NVFP4 (ModelOpt mixed precision); the Hub repo's gguf/ folder is not downloaded
Engine image https://github.com/Morrowmake/vllm-cmp170hx/pkgs/container/vllm-cmp170hx
ghcr.io/morrowmake/vllm-cmp170hx@sha256:5b45c4d6feafefe2eba962375b0a9a0b64fd22b9e05abb4d1966f14a32a2db79 on fork 18305948acba (tag qwen38-recipe-1.0.0) + bind-mounted patches/mimo_v2_omni.py (vLLM #58235)
Reference https://github.com/ElectroHobo/mimo-v26-flash-cmp170hx-recipe
release 0.2.0 = commit c42dbc87cdfd53866d4795d414dcef41bbd9d6c1 (tag v0.2.0, HEAD at ingest) - start.sh, bench/cmp_recipe_bench.py, docs/results.md, docs/known-limits.md, patches/

Revision notes

Community ingest of ElectroHobo's recipe (MIT, (c) 2026 Applied Integrators (ElectroHobo) + Morrowmake for the start.sh/serve.sh script set), listed by Morrowmake 2026-10-19 as the fork's first community recipe; the recipe pins the exact fork commit + image digest per Morrowmake's community-recipe rules and states Morrowmake has not tested/endorsed it. All perf numbers are the author's own measurements (claimed_by=submitter T0, bench harness in-repo) - import via POST /api/v1/benchmarks to upgrade. Measured on x16-capacitor-modded cards behind one PCIe switch with P2P; x4-link performance NOT measured (start.sh warns on narrower links). Sibling recipe mimo-v2-6-flash-rl-omni-on-4x-cmp-170hx-pp4-official-fp8-three-t stays (different quant+drafter+layout).

Revision history

f07639da48 recipe: update mimo-v2-6-flash-rl-309b-moe-nvfp4-4x-cmp-170hx-tp4-dflash-1m-ctx (r2: /kv_pool_tokens) 10 d ago diff
d5a2d27716 recipe: create mimo-v2-6-flash-rl-309b-moe-nvfp4-4x-cmp-170hx-tp4-dflash-1m-ctx 11 d ago

Fork

Export

recipe.yaml (raw)