Qwen3.8-Flash-Next Uncensored AWQ-g32 on 3x CMP 170HX — PP3, 1M ctx, NVMe PLE, MTP-4 (givisok)

IMAGE=vllm-sm80:q38fn FORK=/root/src/vllm-sm80 ./run-qwen38fn.sh

env: 'NCCL_P2P_DISABLE=1 NCCL_IB_DISABLE=1 NCCL_DEBUG=WARN (P2P hangs on these cards) | VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 (1M > checkpoint native 262144) | VLLM_PP_LAYER_PARTITION=17,17,14 (1 layer off the last rank; rollback 17,16,15) | VLLM_PLE_STAGED=1 VLLM_PLE_MMAP=0 VLLM_PLE_SYNC=1 (sync closes the fp8-PLE H2D staging race) | VLLM_QWEN4_QSA_TOPK_CANONICAL=1 VLLM_QWEN_QSA_TOPK_SORTED=1 (QSA top-k tie fix) | VLLM_QWEN4_ROUTER_FP32=1 VLLM_QWEN4_MTP_PER_STREAM_NORM=0 | VLLM_QWEN4EXP_PP_INT8_HOP=1 (g4bwy 0006 port) | VLLM_MTP_DRAFT_HEAD_PATH=/models/qwen38fn-awq-g32/draft-lm-head-w4a16-g128.safetensors | VLLM_MIXED_PREFILL_CHUNK=864 (FN12 bounded prefill) | VLLM_SWA_BLOCK_CAP=432 VLLM_SWA_NO_PREFIX_CACHE=1 (dflash-lane knobs, inert on the mtp lane) | VLLM_PLE_PREFILL_CHECKPOINT=1 | QWEN4EXP_PLE_MODEL_DIR=/models/qwen38fn-awq-g32 | HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1 | PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True PYTORCH_NVML_BASED_CUDA_CHECK=1 VLLM_WORKER_MULTIPROC_METHOD=spawn'
notes: 'run-qwen38fn.sh (default CONFIG=configs/awq-1m-fp8-ssmbf16-mtp4.yml, SPEC_MODE=mtp, since 2026-11-02) execs docker run -d --gpus all --ipc=host --ulimit memlock=-1 --restart unless-stopped -p 0.0.0.0:8080:8000 with /home/models ro at /models, the config yml ro at /opt/stack.yml, the fork vllm/ tree bind-mounted over the image dist-packages, --entrypoint vllm vllm-sm80:q38fn serve --config /opt/stack.yml. All serve args live in the yml: model /models/qwen38fn-awq-g32, served-model-name qwen3.8, api-key set on server, host 0.0.0.0 port 8000, distributed-executor-backend mp, tensor-parallel-size 1, pipeline-parallel-size 3, disable-custom-all-reduce true, dtype bfloat16, max-model-len 1048576, block-size 1728, max-num-seqs 16, max-num-batched-tokens 8192, long-prefill-token-threshold 2048, gpu-memory-utilization 0.95, kv-cache-dtype fp8_e4m3, enable-prefix-caching, prefix-cache-retention-interval 17280, prefix-match-unit 16, enable-chunked-prefill, mamba-cache-mode align, mamba-ssm-cache-dtype bfloat16, async-scheduling true, compilation-config cudagraph_mode FULL_AND_PIECEWISE cudagraph_capture_sizes [1,2,5,10,15,20,25,30,40,50,60,70,80], enable-flashinfer-autotune false, speculative-config {method mtp, model /models/qwen38fn-awq-g32/mtp-fp8, num_speculative_tokens 4}, chat-template /opt/chat_template.jinja (checkpoint chat_template.jinja), trust-request-chat-template, enable-auto-tool-choice, tool-call-parser qwen3_xml, reasoning-parser qwen3, default-chat-template-kwargs {enable_thinking true, preserve_thinking true, reasoning_effort xhigh (xhigh/medium/low only)}, hf-overrides text_config.layer_types [linear_attention x3, qwen_sparse_attention] x12 and rope_parameters {rope_type yarn, factor 4.0, original_max_position_embeddings 262144, rope_theta 10000000, partial_rotary_factor 0.25, mrope_interleaved true, mrope_section [11,11,10]}. CAVEAT (verbatim from the yml): the disk KV tier (kv-disk) must be wiped when swapping model checkpoints — entries are content-addressed by tokens, not weights; the kv-transfer-config tiering block itself is REMOVED (fork OffloadingConnector corrupted multi-chunk cached prefixes — word-salad, ablation 5/5 garbage vs 6/6 clean). Rollback rungs: MTP-3 CONFIG=configs/awq-1m-fp8-ssmbf16.yml; fast code lane valid to depth 262,144 only: SPEC_MODE=dflash (k5+bf16 mamba, code 253-257 solo; drafter has no rope_scaling, speculation dies past the YaRN boundary at ~47 tok/s). Host: Proxmox LXC CT101 Debian, swap 0, 3 cards passed through, memlock unlimited.'

Fetch https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-uncensored-awq-g32-on-3x-cmp-170hx-pp3-1m-ctx.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.

curl -fsSL https://localairecipes.com/api/v1/recipes/qwen3-8-flash-next-uncensored-awq-g32-on-3x-cmp-170hx-pp3-1m-ctx.json