# mistralai/Ministral-3-8B-Reasoning-2512 — vLLM serving recipe

> Ministral 3 Reasoning family (3B/8B/14B) with BF16 weights, vision support, and 256K context

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16) · features `tool_calling`, `reasoning`

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/mistralai/Ministral-3-8B-Reasoning-2512
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/mistralai/Ministral-3-8B-Reasoning-2512.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`, `3b`, `14b`
- `strategy`: `single_node_tp`, `multi_node_tp` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `tool_calling`, `reasoning`, `text_only`, `encoder_parallel` (default: `tool_calling`, `reasoning`)

## Install

Requires vLLM >= 0.11.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve mistralai/Ministral-3-8B-Reasoning-2512 \
  --tokenizer_mode mistral \
  --config_format mistral \
  --load_format mistral \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser mistral \
  --reasoning-parser mistral
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest mistralai/Ministral-3-8B-Reasoning-2512 \
  --tokenizer_mode mistral \
  --config_format mistral \
  --load_format mistral \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser mistral \
  --reasoning-parser mistral
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

Ministral-3 Reasoning comes with BF16 weights in 3 sizes:

- 3B (tied embeddings)
- 8B, 14B (independent embeddings/outputs)

Each variant has vision support and 256K max context. On GB200, we observe significant
speed-ups with NVFP4 Marlin fallback for older GPUs.

### Prerequisites

- Hardware: 1x H200 (3B/8B), 2x H200 recommended for 14B with full context
- vLLM >= 0.11.0

#### Install vLLM

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

### Launch command

3B or 8B on 1x H200:

```bash
vllm serve mistralai/Ministral-3-8B-Reasoning-2512 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral \
  --enable-auto-tool-choice --tool-call-parser mistral \
  --reasoning-parser mistral
```

14B on 2x H200:

```bash
vllm serve mistralai/Ministral-3-14B-Reasoning-2512 \
  --tensor-parallel-size 2 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral \
  --enable-auto-tool-choice --tool-call-parser mistral \
  --reasoning-parser mistral
```

Key flags:
- `enable-auto-tool-choice`: required for tool usage
- `tool-call-parser mistral`: required for tool usage
- `reasoning-parser mistral`: required to extract reasoning content

### Client Usage

Streaming reasoning + answer:

```python
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
stream = client.chat.completions.create(
    model="mistralai/Ministral-3-8B-Reasoning-2512",
    messages=[{"role": "user", "content": "Solve: use 2,5,6,3 to make 24."}],
    stream=True, temperature=0.7, top_p=0.95, max_tokens=262144,
)
for chunk in stream:
    delta = chunk.choices[0].delta
    # vLLM 0.11.0 uses reasoning_content; 0.11.1+ uses reasoning.
    rc = getattr(delta, "reasoning", getattr(delta, "reasoning_content", None))
    if rc:
        print(rc, end="", flush=True)
    if delta.content:
        print(delta.content, end="", flush=True)
```

### Troubleshooting

- OOM on 14B: use `--tensor-parallel-size 2` or lower `--max-model-len`.

### References

- [Ministral-3-8B-Reasoning](https://huggingface.co/mistralai/Ministral-3-8B-Reasoning-2512)
- [Ministral-3-3B-Reasoning](https://huggingface.co/mistralai/Ministral-3-3B-Reasoning-2512)
- [Ministral-3-14B-Reasoning](https://huggingface.co/mistralai/Ministral-3-14B-Reasoning-2512)