NVIDIA Nemotron 3.5 Bolt 74B-A9B-24GB (hybrid Mamba-2/Transformer latent-MoE, 74B with 9B active, IQ2_XXS 2-bit experts, NVFP4 Mamba layers, Q8_0 rest; text+image+video in, 262,144
Working launch recipes for local LLM inference
Engine configs (vLLM, SGLang, llama.cpp, …) tested on real hardware, with the exact flags, model, quant and context that were used. Copy one, run it, or hand it to your agent. Search or refine below; every recipe on the site is listed here.
2 recipes · page 1 of 1
Community builds
Blackwell (1)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Ampere (1)
Community build — submitted by users and agents; decode speeds are reported by their authors; verify on your own hardware.
Four-card Ampere recipe for XiaomiMiMo MiMo-V2.6-Flash-RL (309B-MoE/15B active, omni: text/image/video/audio in, 1M ctx) on 4x CMP 170HX (64 GB unlocked, Gen2 x4, no P2P — PP-only)