# Qwen/Qwen3.5-27B — vLLM serving recipe

> Qwen3.5 dense multimodal model (27B) with gated delta networks hybrid attention, MTP, and 262K context

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`, `encoder_parallel` · frontend `rust`

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/Qwen/Qwen3.5-27B
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/Qwen/Qwen3.5-27B.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`, `arc_pro_b60`, `arc_pro_b70`
- `variant`: `default`, `fp8`, `gptq_int4`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `spec_decoding`, `text_only`, `encoder_parallel`, `long_context` (default: `tool_calling`, `reasoning`, `encoder_parallel`)

## Install

Requires vLLM >= 0.17.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
export VLLM_USE_RUST_FRONTEND=1
vllm serve Qwen/Qwen3.5-27B \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --mm-encoder-tp-mode data
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_USE_RUST_FRONTEND=1 \
  vllm/vllm-openai:latest Qwen/Qwen3.5-27B \
  --trust-remote-code \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --mm-encoder-tp-mode data
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B) is the flagship dense
model of the Qwen3.5 family. It uses the same gated delta networks hybrid
attention as its MoE siblings, supports vision+text input, and natively serves
262K context. MTP (multi-token prediction) is supported out of the box for
low-latency decoding.

### Prerequisites

- **vLLM version:** >= 0.17.0
- **Hardware (BF16):** 1x H200, 2x H100, 4x Intel Arc Pro B60, 2x Intel Arc Pro B70
- **Hardware (FP8):** single 40 GB GPU (H100/H200/L40S)
- **Hardware (Int4):** single 24 GB GPU

#### Install vLLM

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend=auto
```

#### Docker
```bash
docker pull vllm/vllm-openai-xpu:latest  # Intel XPU (B60 / B70)
```

### Launching the Server

#### Single-GPU FP8

```bash
vllm serve Qwen/Qwen3.5-27B-FP8 \
  --max-model-len 262144 \
  --reasoning-parser qwen3
```

#### BF16 on 2xH100 (TP2)

```bash
vllm serve Qwen/Qwen3.5-27B \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --reasoning-parser qwen3
```

#### MTP speculative decoding

```bash
vllm serve Qwen/Qwen3.5-27B-FP8 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}' \
  --reasoning-parser qwen3
```

#### Text-only (skip vision encoder)

```bash
vllm serve Qwen/Qwen3.5-27B-FP8 \
  --language-model-only \
  --reasoning-parser qwen3 \
  --enable-prefix-caching
```

#### Docker (Intel XPU B60 / B70)

Validated on 4× Intel Arc Pro B60 or 2x Intel Arc Pro B70 (B60 24 GB, B70 32 GB per card) with the
official vLLM XPU image `vllm/vllm-openai-xpu:latest`.

```bash
docker run --device /dev/dri \
  -v /dev/dri/by-path:/dev/dri/by-path --shm-size=16g \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai-xpu:latest \
  Qwen/Qwen3.5-27B \
  --reasoning-parser qwen3 \
  --tensor-parallel-size 4 \
  --max-model-len 8192 \
  --enforce-eager
```

### Client Usage

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
resp = client.chat.completions.create(
    model="Qwen/Qwen3.5-27B",
    messages=[{"role": "user", "content": "Write a haiku about gated delta networks."}],
    max_tokens=256,
)
print(resp.choices[0].message.content)
```

### Troubleshooting

- **CUDA graph / Mamba cache size error:** reduce `--max-cudagraph-capture-size`
  (default 512). See [vLLM PR #34571](https://github.com/vllm-project/vllm/pull/34571).
- **Disable reasoning:** add `--default-chat-template-kwargs '{"enable_thinking": false}'`.
- **Prefix Caching (Mamba):** currently experimental in "align" mode.

### References

- [Model card](https://huggingface.co/Qwen/Qwen3.5-27B)
- [FP8 checkpoint](https://huggingface.co/Qwen/Qwen3.5-27B-FP8)
- [GPTQ-Int4 checkpoint](https://huggingface.co/Qwen/Qwen3.5-27B-GPTQ-Int4)
- [Qwen3.5-397B-A17B recipe](../Qwen3.5-397B-A17B)