# Qwen/QwQ-32B — vLLM serving recipe

> Qwen's 32B dense reasoning model, validated for Intel Xeon 6 CPU execution.

Configuration: hardware `xeon6` (4× Xeon 6) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/Qwen/QwQ-32B
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/Qwen/QwQ-32B.json?hardware=xeon6

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `xeon6`
- `variant`: `default`, `w8a8_int8`, `awq`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `long_context` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.7.3.

Docker image: `vllm/vllm-openai-cpu:latest-x86_64`

## Serve

```bash
vllm serve Qwen/QwQ-32B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser deepseek_r1
```

Docker equivalent:

```bash
docker run --shm-size=16g \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/QwQ-32B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser deepseek_r1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

QwQ-32B is Qwen's 32B dense reasoning model. The CPU model validation table
marks it supported on Intel Xeon 6.

### Prerequisites

- Hardware: Intel Xeon 6 CPUs
- vLLM CPU >= 0.8.5 for this validated Xeon 6 configuration

#### pip (Intel Xeon 6 CPUs)

For Intel and AMD x86 CPUs, follow the
[CPU pre-built wheels](https://docs.vllm.ai/en/latest/getting_started/installation/cpu/#pre-built-wheels)
installation instructions.

#### Docker (Intel Xeon 6 CPUs)

```bash
docker pull vllm/vllm-openai-cpu:latest-x86_64
```

### Intel Xeon 6

```bash
vllm serve Qwen/QwQ-32B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser deepseek_r1
```

Docker (the image entrypoint is `vllm serve`):

```bash
docker run \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-cpu:latest-x86_64 Qwen/QwQ-32B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --reasoning-parser deepseek_r1
```

W8A8 INT8:

```bash
vllm serve RedHatAI/QwQ-32B-quantized.w8a8
```

The example uses TP=1. Adjust tensor parallelism based on system topology;
topology-specific CPU binding is intentionally not hard-coded.

### Runtime and Platform Tuning

The following settings are intentionally not prescribed as portable CPU model
defaults because they depend on the workload, hardware, or runtime environment:

- `--max-num-batched-tokens`: scheduler/throughput tuning.
- `--max-num-seqs`: concurrency and scheduler-capacity tuning.
- `--gpu-memory-utilization`: platform memory-budget tuning.
- `--no-enable-prefix-caching`: workload/benchmark cache-policy tuning.
- `VLLM_ENGINE_ITERATION_TIMEOUT_S`: operational runtime timeout.

These settings may still appear in validated hardware-specific overrides. Tune
them at deployment time based on platform resources, workload shape, and
latency/throughput goals.

### References

- [Model card](https://huggingface.co/Qwen/QwQ-32B)
- [W8A8 variant](https://huggingface.co/RedHatAI/QwQ-32B-quantized.w8a8)
- [vLLM reasoning outputs](https://docs.vllm.ai/en/latest/features/reasoning_outputs.html)
- [vLLM tool calling](https://docs.vllm.ai/en/latest/features/tool_calling.html)