# LiquidAI/LFM2.5-VL-450M — vLLM serving recipe

> Liquid AI's smallest vision-language model (450M) — LFM2 hybrid LM backbone plus a SigLIP2 vision tower for image+text chat, light enough for edge GPUs.

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/LiquidAI/LFM2.5-VL-450M
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/LiquidAI/LFM2.5-VL-450M.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp` (multi-node strategies also need `nodes=<n>`)

## Install

Requires vLLM >= 0.23.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

## Serve

```bash
vllm serve LiquidAI/LFM2.5-VL-450M \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest LiquidAI/LFM2.5-VL-450M \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[LFM2.5-VL-450M](https://huggingface.co/LiquidAI/LFM2.5-VL-450M) is the smallest vision-language
model in [Liquid AI's](https://www.liquid.ai/) LFM2.5-VL family — a SigLIP2 vision encoder on
top of the **LFM2 hybrid language backbone** (short-range gated convolution blocks interleaved
with grouped-query attention). It is light enough for edge and on-device image understanding,
served through vLLM's OpenAI-compatible API.

#### Key Features
- **Vision-language**: SigLIP2 vision encoder on top of the LFM2 hybrid language backbone — single- and multi-image prompts.
- **Edge-ready**: ~1 GB of BF16 weights — runs on commodity and on-device GPUs.
- **Hybrid LM backbone**: Gated short convolutions interleaved with grouped-query attention — a smaller KV cache and lower decode latency than a same-size full-attention transformer.
- **32K context**: 32,768-token context window.
- **Native vLLM support**: Served via the `Lfm2VlForConditionalGeneration` architecture.

#### Supported Variants

Vision-Language:
- `LiquidAI/LFM2.5-VL-450M` (450M)
- `LiquidAI/LFM2.5-VL-1.6B` (1.6B)

Text (same LFM2 family):
- Dense: `LiquidAI/LFM2.5-230M`, `LiquidAI/LFM2.5-350M`, `LiquidAI/LFM2.5-1.2B-Instruct`, `LiquidAI/LFM2.5-1.2B-Thinking`, `LiquidAI/LFM2.5-1.2B-JP`, `LiquidAI/LFM2.5-1.2B-JP-202606`, `LiquidAI/LFM2.5-1.2B-Base`
- MoE: `LiquidAI/LFM2.5-8B-A1B`

See the [LFM2.5 usage guide](../../LiquidAI/LFM2.5) for the full family.

### Prerequisites

- **Hardware**: 1× GPU (~1–2 GB VRAM for weights; any modern GPU works).
- **vLLM**: ≥ 0.23.0.

#### pip (NVIDIA CUDA)
```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

### Deployment Configurations

#### Quick Start (Single GPU, BF16)
```bash
vllm serve LiquidAI/LFM2.5-VL-450M
```

#### Multiple Images per Request
```bash
vllm serve LiquidAI/LFM2.5-VL-450M \
  --limit-mm-per-prompt '{"image": 4}' \
  --host 0.0.0.0 --port 8000
```

#### Docker (NVIDIA)
```bash
docker run -itd --name lfm2.5-vl-450m \
  --ipc=host --network host --shm-size 16G --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
    --model LiquidAI/LFM2.5-VL-450M \
    --host 0.0.0.0 --port 8000
```

### Client Usage

#### Image Understanding
Send an image + text turn via the OpenAI chat API. The card recommends temperature `0.1`,
`min_p 0.15`, `repetition_penalty 1.05` (`min_p` and `repetition_penalty` ride in `extra_body`).
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="LiquidAI/LFM2.5-VL-450M",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"}},
            {"type": "text", "text": "What is in this image?"},
        ],
    }],
    temperature=0.1,
    extra_body={"min_p": 0.15, "repetition_penalty": 1.05},
)
print(response.choices[0].message.content)
```

#### Multiple Images
Launch with `--limit-mm-per-prompt '{"image": N}'`, then include several `image_url` blocks in
one message to compare or reason across images.

### Configuration Tips
- At 450M the model fits any GPU; ideal for edge / on-device image understanding.
- `--limit-mm-per-prompt '{"image": N}'` caps images per request (default 1).
- Set `--max-model-len` to match your workload (up to 32K).
- Sampling presets are per-request client defaults — don't bake them into `vllm serve`.

### References
- [Model card](https://huggingface.co/LiquidAI/LFM2.5-VL-450M)
- [LFM2.5 usage guide](../../LiquidAI/LFM2.5)
- [Liquid AI](https://www.liquid.ai/)
- [vLLM supported models](https://docs.vllm.ai/en/latest/models/supported_models.html)