{"slug":"qwen3-8-27b-awq-int4-dspark-on-rtx-3090s-sglang-tp1-tp2-tp4","title":"Qwen3.8-27B AWQ-INT4 + DSpark on RTX 3090s \u2014 SGLang TP1/TP2/TP4","summary":"Validated SGLang serving for the dense Qwen3.8-27B hybrid GDN vision-language model (48 Gated-DeltaNet + 16 gated-attention layers) on RTX 3090 24 GB: one digest-pinned container whose TP env selects tuned 1x/2x/4x presets. AWQ-INT4 W4A16 (Marlin) beats FP8 on Ampere; DSpark speculative decoding (RadixArk 1.36B draft) on 2x/4x with auto-fallback on TP1. Image input everywhere, video on 2x/4x. Author-measured: 136.8 tok/s decode at 8k on 4x (169.8 aggregate at C8), 112.9 on 2x, 43.2 on 1x; up to 128k context on 4x.","license":"CC-BY-4.0","tier":"community","beta":false,"revision":{"no":2,"commit_sha":"26a1363966d9065a310e9ec7caf124206a25f3f2","content_hash":"3fa45739100d87065553d5fb598424a66d4742fb","created_at":"2026-10-20 21:52:36.534482-06","notes":"Ingested by localai-recipes from sybil-solutions/qwen-serve @ 13f8f426e6c7e7de7b141a7e7b060828a660087d (2026-10-12): launch argv transcribed verbatim from qwen3.8/3090-sglang/entrypoint.sh, benchmark rows transcribed from benchmarks/tp{1,2,4}-results.json, image digest + model/drafter revisions from the README + entrypoint pins; W4A16 group-32 quant config verified from the checkpoint config.json at the pinned revision. The submission draft guessed Qwen/Qwen3.8-27B-Instruct-AWQ \u2014 corrected at ingest to the recipe's actual checkpoint cyankiwi/Qwen3.8-27B-AWQ-INT4. The ghcr image is published under the 0xSero account (repo org sybil-solutions). Repo also carries qwen3.6/b70 + qwen3.8/b70 (Arc B70) lanes \u2014 future candidates, out of scope here."},"engine_source":{"type":"docker_image","git_url":"https://github.com/sybil-solutions/qwen-serve","base_engine":"sglang","pip_package":null,"base_version":"git main @ 2e7c85da6 (torch 2.13.0+cu130)","image_digest":"ghcr.io/0xsero/qwen38-3090-sglang@sha256:cc2bfc369c709c53d239ef2af41d4ae2530904975816bb2ca484188008586ac9","install_note":"Digest-pinned image, installable as-is (README: built from source commit 55ab5b9): needs docker + NVIDIA container toolkit and ~26 GB free where /models is mounted (first-run download ~23 GB target + 2.6 GB draft; first boot ~5 min kernel JIT). Pinned stack: sglang git 2e7c85da6, torch 2.13.0+cu130, triton 3.7.1, flashinfer 0.6.17 (cccl guard patched for nvcc 13.3.73), uv + ubuntu:24.04 digest-pinned. TP1/TP2 disable prefill CUDA graphs and cap --cuda-graph-max-bs (capture OOM); TP4 runs full graphs. DSpark auto-disabled on TP=1.","pinned_commit":"13f8f426e6c7e7de7b141a7e7b060828a660087d"},"credit":[{"name":"sybil-solutions / 0xSero (SGLang presets, image build, benchmarks)","url":"https://github.com/sybil-solutions/qwen-serve"},{"name":"cyankiwi (AWQ-INT4 W4A16 checkpoint)","url":"https://huggingface.co/cyankiwi/Qwen3.8-27B-AWQ-INT4"},{"name":"RadixArk (DSpark speculative drafter)","url":"https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark"}],"author":{"name":"localai-recipes","isSite":true},"uploaded_by":"localai-recipes","config":{"launch":["docker","run","-d","--name","qwen38-3090","--gpus","all","--shm-size","32g","-p","8000:8000","-v","$PWD/models:/models","-e","TP=4","ghcr.io/0xsero/qwen38-3090-sglang@sha256:cc2bfc369c709c53d239ef2af41d4ae2530904975816bb2ca484188008586ac9"],"runtime":{"env":"TP=1|2|4 (tensor parallel preset, default 1) | DSPARK=1|0 (default 1; entrypoint auto-disables on TP1 \u2014 draft+target=21.7 GB leaves <1 GB KV on one 24 GB card) | PORT=8000 | MODELS_DIR=/models | weights download pinned revisions cyankiwi/Qwen3.8-27B-AWQ-INT4@63768c1 + RadixArk/Qwen3.8-27B-DSpark@85ef153 into /models, trusted only after the entrypoint records the exact downloaded revision","notes":"docker/entrypoint.sh @ 13f8f42 resolves the server argv internally: python3 -m sglang.launch_server --model-path /models/Qwen3.8-27B-AWQ-INT4 --tp {TP} --chunked-prefill-size 4096 --mamba-radix-cache-strategy extra_buffer plus per-preset flags \u2014 TP1+DSpark: --mem-fraction-static 0.92 --max-mamba-cache-size 4 --disable-prefill-cuda-graph --cuda-graph-max-bs 4 | TP1 plain: 0.92 / 8 / bs 8 | TP2+DSpark: 0.85 / 16 / bs 16 | TP2 plain: 0.85 / 32 / bs 16 | TP4+DSpark: --mem-fraction-static 0.72 (full graphs) | TP4 plain: 0.80; DSpark adds --speculative-algorithm DSPARK --speculative-draft-model-path /models/Qwen3.8-27B-DSpark --speculative-dspark-block-size 7 --speculative-draft-model-quantization unquant. --mamba-radix-cache-strategy extra_buffer is REQUIRED for the hybrid GDN arch (wrong strategies deadlock CUDA graph capture); the mamba state pool sizes concurrency (TP4 max 9 concurrent despite the 275,018-token KV pool; TP2 48). Max context: 8k (TP1) / 64k (TP2) / 128k (TP4). TP1 DSpark possible but impractical (KV ~1 GB). Entry-point preset logic is CI-tested (test/entrypoint-test.sh)."}},"models":[{"slug":"qwen3-8-27b-awq-int4","name":"Qwen3.8-27B-AWQ-INT4","hf_repo":"cyankiwi/Qwen3.8-27B-AWQ-INT4","quant":"awq-w4a16-g32"}],"hardware":[{"slug":"rtx-3090","vendor":"nvidia","card":"RTX 3090","card_family":"rtx-3090","vram_gb":24,"device_count":1},{"slug":"rtx-3090-x2","vendor":"nvidia","card":"RTX 3090","card_family":"rtx-3090","vram_gb":24,"device_count":2},{"slug":"rtx-3090-x4","vendor":"nvidia","card":"RTX 3090","card_family":"rtx-3090","vram_gb":24,"device_count":4}],"generation":"ampere","tags":["awq","dspark","multi-gpu","speculative"],"tasks":["chat","code","vision"],"aa":{"aa_index":34,"aa_provenance":"published","aa_source_url":"https://artificialanalysis.ai/models/qwen3-8-27b","aa_index_version":null,"aa_methodology_version":"v4.3.2","aa_published_fetched_at":"2026-10-07 18:00:00-06","aa_family_slug":"qwen-3-8-27b","aa_resolved_slug":"qwen-3-8-27b"},"performance":[{"claimed_by":"submitter","metrics":"{\"notes\": \"Author-measured tables transcribed from qwen3.8/3090-sglang/benchmarks/tp{1,2,4}-results.json @ 13f8f42 (RTX 3090 24 GB each, PCIe 3.0 x16, no NVLink, driver 610.x, CUDA 13 runtime). Coding-flavored prompts with unique nonce (radix-cache defeated), deterministic decoding, 256-token measurement horizon, streaming usage; decode = aggregate tok/s across concurrent streams. TP1 rows beyond 8k and TP2 128k rows truncated (KV pool 14,775 / 118,693 < prompt) and omitted. DSpark accept length 2.8-4.1 on greedy coding content (AWQ target), ~1.7 on sampled prose. Prefill sustains 1.3-1.6k tok/s on TP2/TP4 across context lengths.\", \"tok_s\": 136.80000000000001136868377216160297393798828125, \"points\": [{\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 5530, \"concurrency\": 1, \"context_len\": 8192, \"decode_tok_s\": 136.80000000000001136868377216160297393798828125, \"prefill_tok_s\": 1580.799999999999954525264911353588104248046875}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 5850, \"concurrency\": 2, \"context_len\": 8192, \"decode_tok_s\": 103.7999999999999971578290569595992565155029296875, \"prefill_tok_s\": 1495.200000000000045474735088646411895751953125}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 6460, \"concurrency\": 4, \"context_len\": 8192, \"decode_tok_s\": 80.5, \"prefill_tok_s\": 1353.59999999999990905052982270717620849609375}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 7740, \"concurrency\": 8, \"context_len\": 8192, \"decode_tok_s\": 60.5, \"prefill_tok_s\": 1130}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 10770, \"concurrency\": 1, \"context_len\": 16384, \"decode_tok_s\": 129.400000000000005684341886080801486968994140625, \"prefill_tok_s\": 1615.799999999999954525264911353588104248046875}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 11960, \"concurrency\": 2, \"context_len\": 16384, \"decode_tok_s\": 97.099999999999994315658113919198513031005859375, \"prefill_tok_s\": 1455.09999999999990905052982270717620849609375}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 12710, \"concurrency\": 4, \"context_len\": 16384, \"decode_tok_s\": 66.900000000000005684341886080801486968994140625, \"prefill_tok_s\": 1368.5}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 15240, \"concurrency\": 8, \"context_len\": 16384, \"decode_tok_s\": 45.60000000000000142108547152020037174224853515625, \"prefill_tok_s\": 1141.700000000000045474735088646411895751953125}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 22000, \"concurrency\": 1, \"context_len\": 32768, \"decode_tok_s\": 127.099999999999994315658113919198513031005859375, \"prefill_tok_s\": 1576.5}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 23210, \"concurrency\": 2, \"context_len\": 32768, \"decode_tok_s\": 93.400000000000005684341886080801486968994140625, \"prefill_tok_s\": 1494.40000000000009094947017729282379150390625}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 25540, \"concurrency\": 4, \"context_len\": 32768, \"decode_tok_s\": 59.60000000000000142108547152020037174224853515625, \"prefill_tok_s\": 1358}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 26680, \"concurrency\": 8, \"context_len\": 32768, \"decode_tok_s\": 39.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1300.09999999999990905052982270717620849609375}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 46340, \"concurrency\": 1, \"context_len\": 65536, \"decode_tok_s\": 113.900000000000005684341886080801486968994140625, \"prefill_tok_s\": 1494.90000000000009094947017729282379150390625}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 48570, \"concurrency\": 2, \"context_len\": 65536, \"decode_tok_s\": 100.599999999999994315658113919198513031005859375, \"prefill_tok_s\": 1426.200000000000045474735088646411895751953125}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 50320, \"concurrency\": 4, \"context_len\": 65536, \"decode_tok_s\": 55, \"prefill_tok_s\": 1376.700000000000045474735088646411895751953125}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 50610, \"concurrency\": 8, \"context_len\": 65536, \"decode_tok_s\": 38, \"prefill_tok_s\": 1368.700000000000045474735088646411895751953125}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 98880, \"concurrency\": 1, \"context_len\": 131072, \"decode_tok_s\": 105, \"prefill_tok_s\": 1400.200000000000045474735088646411895751953125}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 102090, \"concurrency\": 2, \"context_len\": 131072, \"decode_tok_s\": 74.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1356.09999999999990905052982270717620849609375}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 103480, \"concurrency\": 4, \"context_len\": 131072, \"decode_tok_s\": 47, \"prefill_tok_s\": 1337.90000000000009094947017729282379150390625}, {\"notes\": \"4x RTX 3090 (TP4, DSpark) \u2014 production config\", \"ttft_ms\": 103480, \"concurrency\": 8, \"context_len\": 131072, \"decode_tok_s\": 29.300000000000000710542735760100185871124267578125, \"prefill_tok_s\": 1337.90000000000009094947017729282379150390625}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 5900, \"concurrency\": 1, \"context_len\": 8192, \"decode_tok_s\": 112.900000000000005684341886080801486968994140625, \"prefill_tok_s\": 1484.09999999999990905052982270717620849609375}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 6130, \"concurrency\": 2, \"context_len\": 8192, \"decode_tok_s\": 94.7000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1426.299999999999954525264911353588104248046875}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 7580, \"concurrency\": 4, \"context_len\": 8192, \"decode_tok_s\": 79.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1153.700000000000045474735088646411895751953125}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 10020, \"concurrency\": 8, \"context_len\": 8192, \"decode_tok_s\": 73.2999999999999971578290569595992565155029296875, \"prefill_tok_s\": 873}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 11960, \"concurrency\": 1, \"context_len\": 16384, \"decode_tok_s\": 103.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1454.5}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 12730, \"concurrency\": 2, \"context_len\": 16384, \"decode_tok_s\": 73.7999999999999971578290569595992565155029296875, \"prefill_tok_s\": 1365.700000000000045474735088646411895751953125}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 14580, \"concurrency\": 4, \"context_len\": 16384, \"decode_tok_s\": 67.400000000000005684341886080801486968994140625, \"prefill_tok_s\": 1193.40000000000009094947017729282379150390625}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 17300, \"concurrency\": 8, \"context_len\": 16384, \"decode_tok_s\": 65.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1005.1000000000000227373675443232059478759765625}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 25040, \"concurrency\": 1, \"context_len\": 32768, \"decode_tok_s\": 91.5, \"prefill_tok_s\": 1385.299999999999954525264911353588104248046875}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 26340, \"concurrency\": 2, \"context_len\": 32768, \"decode_tok_s\": 79.5, \"prefill_tok_s\": 1316.799999999999954525264911353588104248046875}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 28920, \"concurrency\": 4, \"context_len\": 32768, \"decode_tok_s\": 62, \"prefill_tok_s\": 1199.299999999999954525264911353588104248046875}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 31540, \"concurrency\": 8, \"context_len\": 32768, \"decode_tok_s\": 65.099999999999994315658113919198513031005859375, \"prefill_tok_s\": 1099.59999999999990905052982270717620849609375}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 53440, \"concurrency\": 1, \"context_len\": 65536, \"decode_tok_s\": 100.900000000000005684341886080801486968994140625, \"prefill_tok_s\": 1295.90000000000009094947017729282379150390625}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 56490, \"concurrency\": 2, \"context_len\": 65536, \"decode_tok_s\": 73.099999999999994315658113919198513031005859375, \"prefill_tok_s\": 1226.09999999999990905052982270717620849609375}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 58460, \"concurrency\": 4, \"context_len\": 65536, \"decode_tok_s\": 73.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1184.799999999999954525264911353588104248046875}, {\"notes\": \"2x RTX 3090 (TP2, DSpark)\", \"ttft_ms\": 61080, \"concurrency\": 8, \"context_len\": 65536, \"decode_tok_s\": 59.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1133.90000000000009094947017729282379150390625}, {\"notes\": \"1x RTX 3090 (TP1, no DSpark \u2014 weight-bandwidth-bound)\", \"ttft_ms\": 7030, \"concurrency\": 1, \"context_len\": 8192, \"decode_tok_s\": 43.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 1244.299999999999954525264911353588104248046875}, {\"notes\": \"1x RTX 3090 (TP1, no DSpark \u2014 weight-bandwidth-bound)\", \"ttft_ms\": 10360, \"concurrency\": 2, \"context_len\": 8192, \"decode_tok_s\": 43.2000000000000028421709430404007434844970703125, \"prefill_tok_s\": 843.700000000000045474735088646411895751953125}, {\"notes\": \"1x RTX 3090 (TP1, no DSpark \u2014 weight-bandwidth-bound)\", \"ttft_ms\": 16160, \"concurrency\": 4, \"context_len\": 8192, \"decode_tok_s\": 43.10000000000000142108547152020037174224853515625, \"prefill_tok_s\": 541}, {\"notes\": \"1x RTX 3090 (TP1, no DSpark \u2014 weight-bandwidth-bound)\", \"ttft_ms\": 28260, \"concurrency\": 8, \"context_len\": 8192, \"decode_tok_s\": 43.10000000000000142108547152020037174224853515625, \"prefill_tok_s\": 309.30000000000001136868377216160297393798828125}], \"ttft_ms\": 5530, \"context_len\": 8192}"}],"perf_points":[{"id":1640,"context_len":8192,"prefill_tok_s":1580.799999999999954525264911353588104248046875,"decode_tok_s":136.80000000000001136868377216160297393798828125,"ttft_ms":5530,"concurrency":1,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1641,"context_len":8192,"prefill_tok_s":1495.200000000000045474735088646411895751953125,"decode_tok_s":103.7999999999999971578290569595992565155029296875,"ttft_ms":5850,"concurrency":2,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1642,"context_len":8192,"prefill_tok_s":1353.59999999999990905052982270717620849609375,"decode_tok_s":80.5,"ttft_ms":6460,"concurrency":4,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1643,"context_len":8192,"prefill_tok_s":1130,"decode_tok_s":60.5,"ttft_ms":7740,"concurrency":8,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1644,"context_len":16384,"prefill_tok_s":1615.799999999999954525264911353588104248046875,"decode_tok_s":129.400000000000005684341886080801486968994140625,"ttft_ms":10770,"concurrency":1,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1645,"context_len":16384,"prefill_tok_s":1455.09999999999990905052982270717620849609375,"decode_tok_s":97.099999999999994315658113919198513031005859375,"ttft_ms":11960,"concurrency":2,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1646,"context_len":16384,"prefill_tok_s":1368.5,"decode_tok_s":66.900000000000005684341886080801486968994140625,"ttft_ms":12710,"concurrency":4,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1647,"context_len":16384,"prefill_tok_s":1141.700000000000045474735088646411895751953125,"decode_tok_s":45.60000000000000142108547152020037174224853515625,"ttft_ms":15240,"concurrency":8,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1648,"context_len":32768,"prefill_tok_s":1576.5,"decode_tok_s":127.099999999999994315658113919198513031005859375,"ttft_ms":22000,"concurrency":1,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1649,"context_len":32768,"prefill_tok_s":1494.40000000000009094947017729282379150390625,"decode_tok_s":93.400000000000005684341886080801486968994140625,"ttft_ms":23210,"concurrency":2,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1650,"context_len":32768,"prefill_tok_s":1358,"decode_tok_s":59.60000000000000142108547152020037174224853515625,"ttft_ms":25540,"concurrency":4,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1651,"context_len":32768,"prefill_tok_s":1300.09999999999990905052982270717620849609375,"decode_tok_s":39.2000000000000028421709430404007434844970703125,"ttft_ms":26680,"concurrency":8,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1652,"context_len":65536,"prefill_tok_s":1494.90000000000009094947017729282379150390625,"decode_tok_s":113.900000000000005684341886080801486968994140625,"ttft_ms":46340,"concurrency":1,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1653,"context_len":65536,"prefill_tok_s":1426.200000000000045474735088646411895751953125,"decode_tok_s":100.599999999999994315658113919198513031005859375,"ttft_ms":48570,"concurrency":2,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1654,"context_len":65536,"prefill_tok_s":1376.700000000000045474735088646411895751953125,"decode_tok_s":55,"ttft_ms":50320,"concurrency":4,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1655,"context_len":65536,"prefill_tok_s":1368.700000000000045474735088646411895751953125,"decode_tok_s":38,"ttft_ms":50610,"concurrency":8,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1656,"context_len":131072,"prefill_tok_s":1400.200000000000045474735088646411895751953125,"decode_tok_s":105,"ttft_ms":98880,"concurrency":1,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1657,"context_len":131072,"prefill_tok_s":1356.09999999999990905052982270717620849609375,"decode_tok_s":74.2000000000000028421709430404007434844970703125,"ttft_ms":102090,"concurrency":2,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1658,"context_len":131072,"prefill_tok_s":1337.90000000000009094947017729282379150390625,"decode_tok_s":47,"ttft_ms":103480,"concurrency":4,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"},{"id":1659,"context_len":131072,"prefill_tok_s":1337.90000000000009094947017729282379150390625,"decode_tok_s":29.300000000000000710542735760100185871124267578125,"ttft_ms":103480,"concurrency":8,"claimed_by":"submitter","notes":"4x RTX 3090 (TP4, DSpark) \u2014 production config"}],"agent":{"prompt":"Fetch https://localairecipes.com/api/v1/recipes/qwen3-8-27b-awq-int4-dspark-on-rtx-3090s-sglang-tp1-tp2-tp4.json and apply the inference recipe it describes (engine flags, model, quant, context) to my setup. Ask before overwriting any existing config.","curl":"curl -fsSL https://localairecipes.com/api/v1/recipes/qwen3-8-27b-awq-int4-dspark-on-rtx-3090s-sglang-tp1-tp2-tp4.json"},"requirements":{"requires":["Docker (or a compatible OCI container runtime)","NVIDIA Container Toolkit (nvidia-container-toolkit) - exposes host GPUs to the container"],"driver_floor":{"cuda":null,"linux_min":null,"text":"unknown - check the image tag","source":"NVIDIA CUDA Toolkit Release Notes - driver compatibility tables (https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/, fetched 2026-10-03)"},"placeholders":["Hardware-bound values in this recipe (device counts, bind-mount paths, utilization, ports) describe the author's machine, not yours - map them to your own rig before launching. Keep the engine flags, model, quant, and parallelism strategy; adjust paths and ports to the hardware you actually have.","GPU identity: list your GPUs with `nvidia-smi -L` and substitute your own GPU UUIDs/indices wherever the recipe pins the author's.","Model directory: bind-mount paths are author-side - point them at your own mount where the model files live.","Host port: pick a free port on your machine and publish the container there; keep the in-container port unchanged."]},"artifacts":[{"kind":"model_weights","pointer_type":"hf_model","pointer":"https://huggingface.co/cyankiwi/Qwen3.8-27B-AWQ-INT4","pin":"63768c10df38c0395e12ef49edac1bd539eaeeea \u2014 AWQ_REVISION pinned by docker/entrypoint.sh @ 13f8f42; W4A16 group-32 pack-quantized (compressed-tensors), GDN/visual/lm_head layers unquantized"},{"kind":"drafter","pointer_type":"hf_model","pointer":"https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark","pin":"85ef153be924f17ce4bf62726954eeaa4a73e854 \u2014 DRAFT_REVISION pinned by docker/entrypoint.sh @ 13f8f42; 1.36B DSpark draft (DSparkDraftModel), served unquant"},{"kind":"engine_image","pointer_type":"docker_digest","pointer":"https://ghcr.io/0xsero/qwen38-3090-sglang","pin":"sha256:cc2bfc369c709c53d239ef2af41d4ae2530904975816bb2ca484188008586ac9 \u2014 built from source commit 55ab5b924c3d2176c65bf1132d6c2a5d51ab394e (README pin note)"},{"kind":"reference","pointer_type":"git_commit","pointer":"https://github.com/sybil-solutions/qwen-serve","pin":"13f8f426e6c7e7de7b141a7e7b060828a660087d @ HEAD 'fix(qwen3.8-b70): publish the image under the repository's own package' \u2014 qwen3.8/3090-sglang/ is the recipe source: entrypoint.sh argv, Dockerfile stack pins, benchmarks/tp{1,2,4}-results.json raw rows"}],"links":{"self":"/api/v1/recipes/qwen3-8-27b-awq-int4-dspark-on-rtx-3090s-sglang-tp1-tp2-tp4.json","alternate":"/r/qwen3-8-27b-awq-int4-dspark-on-rtx-3090s-sglang-tp1-tp2-tp4","describedby":"/openapi.json"},"updated_at":"2026-10-20 21:52:36.534482-06","created_at":"2026-10-20 21:49:15.893929-06","speeds":{"decode_tok_s":127.099999999999994315658113919198513031005859375,"prefill_tok_s":1372.700000000000045474735088646411895751953125,"leaderboard":"/api/v1/leaderboard.json"}}