# internlm/Intern-S1 — vLLM serving recipe

> Intern-S1 vision-language model from Shanghai AI Lab with BF16/FP8 variants and thinking/non-thinking modes

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`, `encoder_parallel`

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/internlm/Intern-S1
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/internlm/Intern-S1.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `fp8`
- `strategy`: `single_node_tp`, `multi_node_tp`, `multi_node_tp_pp`, `multi_node_tep`, `multi_node_dep` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `text_only`, `encoder_parallel` (default: `tool_calling`, `reasoning`, `encoder_parallel`)

## Install

Requires vLLM >= 0.10.2.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve internlm/Intern-S1 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --tool-call-parser internlm \
  --reasoning-parser deepseek_r1 \
  --mm-encoder-tp-mode data
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest internlm/Intern-S1 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --tool-call-parser internlm \
  --reasoning-parser deepseek_r1 \
  --mm-encoder-tp-mode data
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Intern-S1](https://github.com/InternLM/Intern-S1) is a vision-language model developed
by Shanghai AI Laboratory. It supports thinking and non-thinking modes via chat-template
kwargs and ships in BF16 and FP8 variants.

### Prerequisites

- Hardware: 8xH800 (80GB) for BF16, 4xH800 for FP8, or 4-8x MI300X/MI325X/MI355X
- vLLM >= 0.10.2

#### Install vLLM (CUDA)

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

#### Install vLLM (AMD ROCm)

```bash
uv pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.14.1/rocm700
```

### Launch commands

BF16 on 8xH800:

```bash
vllm serve internlm/Intern-S1 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_r1 \
  --tool-call-parser internlm
```

FP8 on 4xH800:

```bash
vllm serve internlm/Intern-S1-FP8 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --enable-auto-tool-choice \
  --reasoning-parser deepseek_r1 \
  --tool-call-parser internlm
```

FP8 on 8xMI300X/MI325X:

```bash
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_MOE=0
vllm serve internlm/Intern-S1-FP8 \
  --trust-remote-code --tensor-parallel-size 8 \
  --enable-auto-tool-choice --reasoning-parser deepseek_r1 --tool-call-parser internlm
```

FP8 on 8xMI355X: set only `VLLM_ROCM_USE_AITER=1` (no need to disable AITER MoE).

### Switching Between Thinking and Non-Thinking Modes

```python
from openai import OpenAI
client = OpenAI(api_key="YOUR_API_KEY", base_url="http://0.0.0.0:8000/v1")
model_name = client.models.list().data[0].id

response = client.chat.completions.create(
    model=model_name,
    messages=[{"role": "user", "content": "9.11 and 9.8, which is greater?"}],
    temperature=0.8, top_p=0.8,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response)
```

### Troubleshooting

- `ValueError: No available memory for the cache blocks.` — add `--gpu-memory-utilization 0.95`.

### References

- [Intern-S1 on Hugging Face](https://huggingface.co/internlm/Intern-S1)
- [Intern-S1-FP8](https://huggingface.co/internlm/Intern-S1-FP8)
- [Intern-S1 GitHub](https://github.com/InternLM/Intern-S1)