Qwen3-30B-A3B (MoE)

Mixture-of-Experts model — 30B total parameters, 3B active (128 experts), BF16. Validated on Ascend 910B3 at TP=2 and TP=4 (with expert parallel) using both vLLM-Ascend and MindIE.

Model identity

Field	Value
Publisher	Qwen (Alibaba)
Architecture	Qwen3 MoE-A3B — 30B total / 3B active (128 experts)
Parameters	30B (3B activated)
Native dtype	BF16
Hugging Face	https://huggingface.co/Qwen/Qwen3-30B-A3B
ModelScope	https://www.modelscope.cn/models/Qwen/Qwen3-30B-A3B

Validated hardware × stack

Platform	Engine	Version	Status
Ascend 910B3 × 2 / × 4 (TP=2/TP=4 + EP)	MindIE	`2.2.RC1` (ATB, auto BF16)	✅ 4-scenario open-loop perf
Ascend 910B3 × 2 / × 4 (TP=2/TP=4 + EP)	vLLM-Ascend	`v0.18.0-openeuler` (EP + `--enforce-eager`)	✅ 4-scenario open-loop perf

NOTE

This model has also been validated as W8A8 with multi-node Prefill/Decode (PD) disaggregation on 910B4 (cross-node KV transfer) — a separate, more complex topology. The recipes on this page are the single-node aggregated deployments.

Deploy

Self-contained ServingRuntime + InferenceService YAMLs:

Config	File
MindIE, TP=2 (rate-1 chat ITL P90 30.8 ms)	`qwen3-30b-a3b-mindie-tp2.yaml`
MindIE, TP=4	`qwen3-30b-a3b-mindie-tp4.yaml`
vLLM-Ascend, TP=4	`qwen3-30b-a3b-vllm-ascend-tp4.yaml`

base=https://raw.githubusercontent.com/alauda/aml-docs/master/docs/en/inference_guide/assets/qwen3-30b-a3b
# edit namespace / image tag / storageUri first, then:
kubectl apply -f $base/qwen3-30b-a3b-mindie-tp2.yaml

WARNING

MindIE requires root and a writable model volume. Keep serving.kserve.io/readonly: "false" on the InferenceService and the root securityContext in the asset. See the Modelcar permission modes in Extend Inference Runtimes.

Benchmark results

Open-loop per-replica, replica=1. Saturation capacity (RPS/replica) by workload (< 1 = the workload does not sustain one request/second/replica on this hardware/stack).

Workload (prompt/output)	MindIE TP=2	MindIE TP=4	vLLM TP=2	vLLM TP=4
Chat `512/256`	5.90	6.48	2.68	2.69
Code `1024/1024`	1.16	1.71	0.13	0.12
RAG `4096/512`	1.07	1.69	0.66	1.24
Long RAG `10240/1536`	0.02	0.14	0	0

Rate-1 latency snapshot, Chat workload (TP=2):

Engine	Achieved RPS	TTFT p90 / mean (ms)	ITL p90 / mean (ms)	TPS mean (tok/s)
MindIE	0.97	115 / 104	30.8 / 29.0	250
vLLM-Ascend	0.83	587 / 510	195.2 / 193.2	215

Full open-loop data (all 22 columns)

Every rate level (1–9) × all four workloads × both engines × TP=2/TP=4, with TTFT / E2E / ITL / TPS at p90 / p95 / p99 / mean. Expand each section:

Full 22-column open-loop sweep — 910B3 × 2 · Qwen3-30B-A3B (MoE) · vllm-ascend v0.18.0-openeuler (EP+eager)

Chat 512/256 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	520	256	587	606	633	510	50,263	50,486	50,624	49,771	195.2	195.8	196.4	193.2	446.9	619.7	1,192.2	214.7
2*	1	2	2	520	256	594	608	638	513	50,271	50,339	50,460	49,857	195.1	195.3	195.6	193.5	918.0	1,279.5	2,449.9	426.7
3*	1	3	3	520	256	883	906	954	721	75,376	75,449	75,568	72,377	292.4	292.8	293.1	281.0	1,220.7	1,692.6	3,160.7	576.6
4*	1	4	4	520	256	23,826	23,927	24,059	9,849	99,240	99,330	99,433	83,105	295.7	295.9	296.5	287.3	1,583.4	2,290.7	4,445.5	686.3
5*	1	5	5	520	256	51,640	51,932	59,937	24,236	127,980	128,316	136,019	98,731	299.8	300.0	300.3	292.1	1,504.4	2,163.1	4,169.3	683.0
6*	1	6	6	520	256	33,556	33,624	33,731	16,269	107,807	107,881	107,962	90,077	293.3	293.6	293.8	289.4	1,539.0	2,223.9	4,341.9	680.3
7*	1	7	7	520	256	846	894	931	720	74,545	74,575	74,686	73,668	289.1	289.2	289.4	286.1	994.4	1,627.6	3,777.0	362.5
8*	1	8	8	520	256	878	913	943	746	74,749	74,784	74,878	74,403	289.8	289.9	290.0	288.8	1,559.8	2,730.7	6,610.4	615.5
9*	1	9	9	520	256	860	882	919	741	73,841	73,897	73,959	73,337	286.4	286.6	286.7	284.7	982.3	1,901.3	4,888.5	362.7

Code 1024/1024 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	1032	1024	680	690	691	585	259,677	260,236	260,690	253,700	253.2	253.9	254.2	247.4	254.7	341.6	639.3	133.9
2*	1	2	2	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
3*	1	3	3	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
4*	1	4	4	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
5*	1	5	5	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
6*	1	6	6	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
7*	1	7	7	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
8*	1	8	8	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
9*	1	9	9	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

RAG 4096/512 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	4104	512	1,114	1,150	1,264	1,003	113,578	113,639	113,723	112,403	220.1	220.3	220.5	218.0	668.0	914.8	1,710.9	319.6
2*	1	2	2	4104	512	1,095	1,124	1,164	988	113,069	113,135	121,106	112,836	219.3	219.4	235.0	218.9	751.1	1,029.5	1,933.8	344.8
3*	1	3	3	4104	512	1,327	1,327	1,327	1,118	111,138	111,138	111,138	110,866	214.9	214.9	214.9	214.8	44.9	59.2	81.2	27.4
4*	1	4	4	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
5*	1	5	5	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
6*	1	6	6	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
7*	1	7	7	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
8*	1	8	8	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
9*	1	9	9	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

Long RAG 10240/1536 (capacity < 1 RPS/replica) — capacity 0: all 9 rate levels errored (requests queue past the TTFT timeout). No sustained throughput.

Full 22-column open-loop sweep — 910B3 × 2 · Qwen3-30B-A3B (MoE) · MindIE 2.2.RC1

Chat 512/256 — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1	1	1	1	520	256	115	121	211	104	7,987	8,321	8,740	7,493	30.8	32.2	33.8	29.0	433.4	567.4	1,024.8	249.7
2	1	2	2	520	256	120	122	128	106	11,189	11,355	11,868	10,775	43.4	44.1	46.1	41.8	922.2	1,237.3	2,259.9	494.5
3	1	3	3	520	256	128	131	138	110	14,937	15,003	15,154	14,680	58.1	58.4	59.0	57.1	1,442.3	1,962.7	3,560.5	730.2
4	1	4	4	520	256	140	142	148	118	20,797	20,811	20,907	20,463	81.1	81.1	81.5	79.8	2,034.6	2,826.4	5,200.6	954.4
5	1	5	5	520	256	150	155	161	123	28,816	28,851	29,006	27,948	112.5	112.6	113.2	109.1	2,300.8	3,093.1	5,377.3	1,159.8
6*	1	6	6	520	256	166	171	182	133	83,254	83,674	84,376	58,071	326.1	327.5	330.3	227.2	2,216.9	2,962.1	5,130.6	1,109.9
7*	1	7	7	520	256	1,580	2,216	3,417	582	73,924	74,667	76,623	68,379	288.5	292.3	300.0	265.9	2,809.3	4,148.7	9,104.9	1,366.0
8*	1	8	8	520	256	2,221	3,051	4,035	865	72,612	73,017	73,606	67,506	284.3	285.8	288.0	261.3	3,259.0	5,053.4	11,726.9	1,504.1
9*	1	9	9	520	256	2,480	3,376	5,083	934	74,598	74,979	75,381	68,848	292.1	293.5	295.1	266.3	3,287.1	5,053.4	11,275.0	1,508.0

Code 1024/1024 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	1032	1024	171	174	195	149	67,941	68,012	68,275	63,188	66.3	66.3	66.6	61.6	1,561.7	2,088.7	3,706.9	793.2
2*	1	2	2	1032	1024	206	446	1,842	219	124,495	125,095	126,046	110,642	121.3	121.7	122.1	107.9	2,487.0	3,344.7	5,809.3	1,193.9
3*	1	3	3	1032	1024	196	206	225	156	133,332	133,843	134,927	116,007	130.1	130.7	131.7	113.2	2,458.6	3,338.6	5,847.4	1,136.6
4*	1	4	4	1032	1024	173	179	193	150	117,735	117,926	118,148	113,941	114.9	115.1	115.4	111.2	1,901.3	2,559.4	4,476.3	941.3
5*	1	5	5	1032	1024	153	154	154	138	115,931	116,024	116,024	115,189	113.2	113.3	113.3	112.5	266.4	352.4	623.8	154.5
6*	1	6	6	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
7*	1	7	7	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
8*	1	8	8	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
9*	1	9	9	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

RAG 4096/512 — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1	1	1	1	4104	512	342	344	353	321	36,934	36,977	37,012	35,559	71.6	71.7	71.8	69.0	851.8	1,160.3	2,138.3	452.2
2*	1	2	2	4104	512	36,932	37,407	46,715	18,209	94,430	94,902	104,125	75,324	114.3	114.8	115.3	111.8	1,158.4	1,736.1	4,211.1	556.5
3*	1	3	3	4104	512	357	360	31,980	1,467	60,538	60,709	90,946	60,881	117.7	117.9	118.4	116.2	837.0	1,159.0	2,171.0	354.9
4*	1	4	4	4104	512	535	756	758	490	58,697	59,699	60,452	50,958	114.2	116.2	117.7	98.8	1,296.9	1,773.5	3,219.0	655.3
5*	1	5	5	4104	512	2,597	2,757	3,008	1,714	58,122	58,926	59,525	51,933	112.5	114.4	115.9	98.3	1,441.3	2,097.2	4,241.0	665.5
6*	1	6	6	4104	512	4,817	5,014	5,366	2,883	58,316	58,984	59,486	53,161	112.1	114.4	115.8	98.4	1,443.8	2,029.2	3,848.0	666.0
7*	1	7	7	4104	512	6,416	6,703	7,209	3,771	58,034	58,604	59,035	53,613	110.8	113.5	114.9	97.5	1,469.1	2,130.2	4,262.5	671.1
8*	1	8	8	4104	512	7,629	8,067	8,500	4,458	58,536	59,040	59,415	54,673	112.1	113.4	115.7	98.3	1,479.2	2,134.5	4,104.0	666.8
9*	1	9	9	4104	512	8,557	9,018	9,476	4,951	57,987	58,432	58,765	54,555	110.8	112.2	114.4	97.1	1,481.9	2,071.3	3,968.1	674.1

Long RAG 10240/1536 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	10248	1536	772	772	772	760	119,253	119,253	119,253	119,211	77.2	77.2	77.2	77.2	25.9	25.9	25.9	25.7
2*	1	2	2	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
3*	1	3	3	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
4*	1	4	4	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
5*	1	5	5	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
6*	1	6	6	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
7*	1	7	7	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
8*	1	8	8	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
9*	1	9	9	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

Full 22-column open-loop sweep — 910B3 × 4 · Qwen3-30B-A3B (MoE) · vllm-ascend v0.18.0-openeuler (EP+eager)

Chat 512/256 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	520	256	598	614	639	510	50,724	50,892	51,050	50,123	196.9	197.4	198.2	194.6	436.0	601.4	1,122.7	214.3
2*	1	2	2	520	256	590	610	646	513	50,437	50,491	50,576	50,093	195.8	196.0	196.2	194.4	896.2	1,267.2	2,465.8	426.2
3*	1	3	3	520	256	874	897	949	714	75,338	75,542	76,015	71,897	292.3	293.3	294.9	279.1	1,247.9	1,748.0	3,320.9	574.1
4*	1	4	4	520	256	22,564	22,849	23,067	9,149	97,950	98,149	98,338	81,977	295.7	295.9	296.2	285.6	1,553.7	2,223.9	4,284.3	689.9
5*	1	5	5	520	256	48,963	49,159	59,766	23,222	124,432	124,804	135,866	96,727	295.8	297.1	298.4	288.2	1,557.5	2,232.2	4,350.9	690.2
6*	1	6	6	520	256	33,290	33,358	33,456	15,773	107,308	107,385	107,463	88,985	292.3	292.5	292.6	287.1	1,495.3	2,143.2	4,140.5	681.8
7*	1	7	7	520	256	887	907	976	753	74,655	74,758	74,826	74,010	289.8	289.9	290.0	287.3	1,400.9	2,100.8	4,204.8	590.0
8*	1	8	8	520	256	900	916	970	764	74,423	74,451	74,494	74,127	288.5	288.6	288.6	287.7	1,002.0	1,650.7	3,782.1	360.9
9*	1	9	9	520	256	884	915	962	762	75,017	75,058	75,121	74,478	290.9	290.9	291.0	289.1	1,536.9	2,392.6	5,102.6	635.7

Code 1024/1024 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	1032	1024	687	716	758	592	261,870	262,405	262,776	256,676	255.4	256.0	256.3	250.3	235.1	316.9	584.3	123.9
2*	1	2	2	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
3*	1	3	3	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
4*	1	4	4	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
5*	1	5	5	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
6*	1	6	6	1032	1024	883	883	883	883	186,360	186,360	186,360	186,360	181.3	181.3	181.3	181.3	5.5	5.5	5.5	5.5
7*	1	7	7	1032	1024	944	944	944	944	186,188	186,188	186,188	186,188	181.1	181.1	181.1	181.1	5.5	5.5	5.5	5.5
8*	1	8	8	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
9*	1	9	9	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

RAG 4096/512 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	4104	512	1,120	1,172	1,329	1,010	113,572	113,636	113,755	112,093	220.3	220.3	220.5	217.4	660.7	903.2	1,688.5	321.3
2*	1	2	2	4104	512	1,163	1,191	1,271	1,052	113,131	113,163	113,227	112,576	219.2	219.3	219.4	218.2	1,391.5	1,938.9	3,646.6	639.9
3*	1	3	3	4104	512	24,790	26,757	28,182	12,671	133,221	135,206	136,637	119,699	212.2	212.2	212.3	209.4	1,296.9	1,797.4	3,360.8	601.9
4*	1	4	4	4104	512	11,696	12,269	12,903	6,657	118,269	118,963	119,721	112,638	208.6	208.8	209.0	207.4	580.9	831.5	1,664.4	285.6
5*	1	5	5	4104	512	3,832	4,042	4,254	2,360	108,559	108,822	109,022	106,352	204.9	205.0	205.0	203.5	121.3	187.8	401.9	49.7
6*	1	6	6	4104	512	1,121	1,121	1,121	1,121	93,526	93,526	93,526	93,526	180.8	180.8	180.8	180.8	5.5	5.5	5.5	5.5
7*	1	7	7	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
8*	1	8	8	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
9*	1	9	9	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

Long RAG 10240/1536 (capacity < 1 RPS/replica) — capacity 0: all 9 rate levels errored (requests queue past the TTFT timeout). No sustained throughput.

Full 22-column open-loop sweep — 910B3 × 4 · Qwen3-30B-A3B (MoE) · MindIE 2.2.RC1

Chat 512/256 — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1	1	1	1	520	256	115	146	234	94	10,248	10,335	10,539	9,169	39.8	40.1	41.0	35.6	433.3	568.9	1,033.1	248.4
2	1	2	2	520	256	101	108	132	87	11,180	11,307	11,505	9,874	43.4	44.0	44.8	38.4	903.4	1,199.7	2,137.8	497.6
3	1	3	3	520	256	105	112	162	89	12,980	13,192	13,594	11,785	50.5	51.5	52.9	45.9	1,368.9	1,817.3	3,193.2	737.7
4	1	4	4	520	256	107	112	145	89	15,199	15,666	15,779	13,890	59.3	61.1	61.5	54.1	1,834.8	2,437.1	4,221.7	979.2
5	1	5	5	520	256	110	114	134	92	17,006	17,136	17,383	16,456	66.3	66.8	67.8	64.2	2,347.8	3,137.1	5,398.1	1,210.2
6	1	6	6	520	256	121	128	216	99	21,815	22,390	23,040	20,760	85.1	87.5	90.0	81.0	2,774.0	3,685.7	6,269.5	1,431.9
7	1	7	7	520	256	125	131	172	101	27,191	27,602	28,048	25,442	106.2	107.8	109.6	99.4	3,164.3	4,177.6	6,990.5	1,635.2
8*	1	8	8	520	256	135	148	199	111	67,578	67,939	68,496	54,263	264.6	266.0	268.2	212.4	3,156.0	4,211.1	7,206.7	1,583.5
9*	1	9	9	520	256	148	156	186	120	65,754	66,780	67,058	60,393	257.4	261.4	262.6	236.4	3,368.9	4,559.0	8,004.4	1,659.7

Code 1024/1024 — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1	1	1	1	1032	1024	135	140	266	117	46,186	46,355	46,474	44,522	45.0	45.2	45.3	43.4	1,691.3	2,267.0	3,975.9	874.8
2*	1	2	2	1032	1024	147	153	198	122	76,956	76,995	77,032	72,959	75.1	75.1	75.2	71.2	3,080.0	4,096.0	6,887.2	1,525.5
3*	1	3	3	1032	1024	158	165	196	127	187,558	187,711	187,750	114,584	183.2	183.4	183.4	111.9	2,621.4	3,625.6	6,459.1	1,151.7
4*	1	4	4	1032	1024	154	161	196	125	205,162	209,423	212,868	161,776	200.5	204.6	208.0	158.0	2,504.1	3,394.6	5,942.9	1,182.3
5*	1	5	5	1032	1024	155	161	184	123	219,823	222,283	226,664	180,215	214.8	217.1	221.5	176.0	2,659.1	3,599.2	6,267.7	1,248.1
6*	1	6	6	1032	1024	151	156	181	122	224,514	232,688	234,988	187,325	219.3	227.4	229.6	183.0	2,818.6	3,813.5	6,589.0	1,324.9
7*	1	7	7	1032	1024	152	159	189	123	238,503	240,573	241,737	198,131	233.0	235.0	236.2	193.6	2,933.1	3,953.3	6,813.4	1,393.0
8*	1	8	8	1032	1024	158	182	455	135	243,807	244,264	244,630	207,207	238.2	238.6	239.0	202.4	3,153.6	4,227.9	7,194.3	1,516.2
9*	1	9	9	1032	1024	150	155	188	125	242,101	242,203	242,339	211,232	236.5	236.6	236.8	206.4	3,271.7	4,428.0	7,623.4	1,423.0

RAG 4096/512 — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1	1	1	1	4104	512	256	262	274	239	23,982	24,299	24,435	22,520	46.4	47.1	47.4	43.6	874.0	1,164.0	2,088.8	473.8
2*	1	2	2	4104	512	268	274	292	244	55,037	55,345	55,494	50,869	107.2	107.9	108.1	99.1	1,646.1	2,211.7	3,887.2	837.9
3*	1	3	3	4104	512	459	715	7,806	413	123,457	123,631	127,791	108,562	240.5	240.9	241.4	211.6	1,818.1	2,464.3	4,378.2	870.6
4*	1	4	4	4104	512	21,857	22,294	22,617	5,192	142,170	144,016	146,093	117,326	240.6	242.2	247.0	219.4	1,771.6	2,441.4	4,490.7	829.5
5*	1	5	5	4104	512	278	287	316	251	118,977	119,424	120,140	108,327	232.3	233.2	234.6	211.5	1,190.2	1,636.5	3,030.6	546.6
6*	1	6	6	4104	512	540	545	554	425	98,071	98,722	99,383	92,079	191.2	192.5	193.9	179.3	885.6	1,199.1	2,240.5	459.5
7*	1	7	7	4104	512	1,450	1,531	1,664	1,008	94,012	94,727	95,298	88,159	182.8	184.7	186.1	170.5	819.5	1,162.8	2,328.9	346.9
8*	1	8	8	4104	512	2,825	2,951	3,164	1,740	94,203	94,631	95,115	89,652	183.0	184.4	185.7	172.0	972.7	1,345.6	2,563.8	478.8
9*	1	9	9	4104	512	3,935	4,158	4,425	2,351	94,260	94,663	95,114	90,226	182.9	183.8	185.7	172.0	958.7	1,334.5	2,641.2	484.0

Long RAG 10240/1536 (capacity < 1 RPS/replica) — units: TTFT / E2E / ITL in ms, TPS in tok/s. * = past saturation (achieved RPS < target rate).

rate	replica	Total RPS	RPS/rep	Mean In	Mean Out	TTFT p90	TTFT p95	TTFT p99	TTFT mean	E2E p90	E2E p95	E2E p99	E2E mean	ITL p90	ITL p95	ITL p99	ITL mean	TPS p90	TPS p95	TPS p99	TPS mean
1*	1	1	1	10248	1536	566	569	576	539	187,723	188,821	190,020	173,963	122.0	122.7	123.4	113.0	410.7	552.6	1,012.0	210.9
2*	1	2	2	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
3*	1	3	3	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
4*	1	4	4	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
5*	1	5	5	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
6*	1	6	6	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
7*	1	7	7	10248	1536	1,149	1,149	1,149	1,149	48,718	48,718	48,718	48,718	31.0	31.0	31.0	31.0	32.3	32.3	32.3	32.3
8*	1	8	8	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
9*	1	9	9	0	0	0	0	0	0	0	0	0	0	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

#Qwen3-30B-A3B (MoE)

#TOC

#Model identity

#Validated hardware × stack

#Deploy

#Benchmark results

#Full open-loop data (all 22 columns)

Qwen3-30B-A3B (MoE)

TOC

Model identity

Validated hardware × stack

Deploy

Benchmark results

Full open-loop data (all 22 columns)