# Qwen/Qwen3-30B-A3B — vLLM serving recipe

> Qwen3 MoE model with 30.5B total and 3.3B active parameters, validated for BF16 serving on Intel Xeon 6 and Intel Arc Pro B70 and W8A8 serving on Ascend A3.

Configuration: hardware `arc_pro_b70` (4× Arc Pro B70) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/Qwen/Qwen3-30B-A3B
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/Qwen/Qwen3-30B-A3B.json?hardware=arc_pro_b70

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `xeon6`, `arc_pro_b70`
- `variant`: `default`, `ascend_w8a8`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `long_context` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.8.5.

Docker image: `vllm/vllm-openai-xpu:latest`

## Serve

```bash
vllm serve Qwen/Qwen3-30B-A3B \
  --tensor-parallel-size 4 \
  --block-size 64 \
  --enforce-eager \
  --no-enable-prefix-caching \
  --disable-sliding-window \
  --max-model-len 9472 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.9 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3
```

Docker equivalent:

```bash
docker run --device /dev/dri \
  -v /dev/dri/by-path:/dev/dri/by-path --shm-size=16g \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-xpu:latest Qwen/Qwen3-30B-A3B \
  --tensor-parallel-size 4 \
  --block-size 64 \
  --enforce-eager \
  --no-enable-prefix-caching \
  --disable-sliding-window \
  --max-model-len 9472 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.9 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Qwen3-30B-A3B is a mixture-of-experts Qwen3 model with 30.5B total parameters
and 3.3B active parameters per token. This recipe includes an Intel Xeon 6 CPU
configuration validated with BF16 serving and an Ascend A3 W8A8 variant.

### Prerequisites

- Hardware: 4x Xeon 6 CPUs, or 4x Intel Arc Pro B70 (32 GB per card).
- vLLM >= 0.8.5

#### pip (Intel Xeon 6 CPUs)

For Intel and AMD x86 CPUs, follow the
[CPU pre-built wheels](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#pre-built-wheels)
installation instructions.

#### Docker (Intel Xeon 6 CPUs)

```bash
docker pull vllm/vllm-openai-cpu:latest-x86_64
```

#### Docker (Intel Arc Pro B70)

```bash
docker pull vllm/vllm-openai-xpu:latest
```

### Intel Xeon 6

Choose the tensor parallel size based on the system topology and deployment
requirements. Add `--tensor-parallel-size <N>` when needed; the recipe does
not prescribe a Xeon 6 TP value or hard-code NUMA node IDs.

```bash
vllm serve Qwen/Qwen3-30B-A3B \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.8
```

Docker:

```bash
docker run \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/Qwen3-30B-A3B \
  --tensor-parallel-size 1 \
  --max-num-batched-tokens 16384 \
  --gpu-memory-utilization 0.8 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser qwen3
```

### Intel Arc Pro B70 (XPU)

Validated on 4x Intel Arc Pro B70 (32 GB per card) with the official vLLM XPU
image `vllm/vllm-openai-xpu:latest`, BF16, TP=4.

```bash
docker run --device /dev/dri \
  -v /dev/dri/by-path:/dev/dri/by-path --shm-size=16g \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-xpu:latest \
  Qwen/Qwen3-30B-A3B \
  --tensor-parallel-size 4 \
  --block-size 64 \
  --enforce-eager \
  --no-enable-prefix-caching \
  --disable-sliding-window \
  --max-model-len 9472 \
  --max-num-batched-tokens 8192 \
  --gpu-memory-utilization 0.9
```

### Client Usage

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
response = client.chat.completions.create(
    model="Qwen/Qwen3-30B-A3B",
    messages=[{"role": "user", "content": "Explain tensor parallelism briefly."}],
)
print(response.choices[0].message.content)
```

### References

- [Model card](https://huggingface.co/Qwen/Qwen3-30B-A3B)
- [Ascend A3 W8A8 checkpoint](https://www.modelscope.cn/models/Eco-Tech/Qwen3-30B-A3B-w8a8)
- [vLLM Ascend](https://github.com/vllm-project/vllm-ascend)
- [vLLM CPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/)