# OpenMOSS-Team/MOSS-Transcribe-Diarize — vLLM serving recipe

> OpenMOSS's 0.9B end-to-end multi-speaker long-audio transcription model with timestamps and speaker labels, served through vLLM's OpenAI-compatible /v1/audio/transcriptions API.

Configuration: hardware `h200` (8× H200) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (BF16)

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/OpenMOSS-Team/MOSS-Transcribe-Diarize
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/OpenMOSS-Team/MOSS-Transcribe-Diarize.json?hardware=h200

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`, `mi300x`, `mi325x`, `mi350x`, `mi355x`
- `variant`: `default`
- `strategy`: `single_node_tp` (multi-node strategies also need `nodes=<n>`)

## Install

Requires vLLM >= 0.27.0.

```bash
uv venv
source .venv/bin/activate
uv pip install -U vllm --torch-backend auto
```

Docker image: `vllm/vllm-openai:latest`

Extra dependencies:

- Audio extras are required for loading and preprocessing audio files.

```bash
uv pip install -U "vllm[audio]" --torch-backend auto
```

## Serve

```bash
vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --trust-remote-code \
  --tensor-parallel-size 1
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --trust-remote-code \
  --tensor-parallel-size 1
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[MOSS-Transcribe-Diarize](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize)
is a 0.9B end-to-end speech-to-text model for long-form multi-speaker
transcription. It uses a Whisper-style audio encoder and a Qwen3-style causal
decoder to generate structured transcripts with text, timestamps, and speaker
labels in a single pass.

Example output:

```text
[0.11][S01] Good morning! [1.03]
[1.11][S02] Morning, let's start the meeting. [2.42]
```

Key capabilities:

- End-to-end multi-speaker transcription without a separate ASR, diarization,
  or alignment pipeline.
- Structured text output with speaker IDs and timestamps.
- Long-audio transcription for meetings, interviews, podcasts, and customer
  support recordings.
- Prompt-based hotwords for names, organizations, product names, project
  codes, and domain vocabulary.

### Prerequisites

Install vLLM with audio dependencies.

```bash
uv venv
source .venv/bin/activate

uv pip install -U "vllm[audio]" --torch-backend auto
```

### Launching with vLLM

```bash
vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --trust-remote-code \
  --served-model-name moss-mtd
```

The model uses custom Hugging Face processor code, so `--trust-remote-code` is
required.

### Client Usage

#### OpenAI Transcription API

```python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

with open("meeting.wav", "rb") as audio_file:
    transcription = client.audio.transcriptions.create(
        model="moss-mtd",
        file=audio_file,
        response_format="diarized_json",
        temperature=0,
    )

print(transcription.text)

for segment in transcription.segments:
    print(
        f"[{segment.start:.2f} - {segment.end:.2f}] "
        f"{segment.speaker}: {segment.text}"
    )
```

#### cURL

```bash
curl http://localhost:8000/v1/audio/transcriptions \
  -F "model=moss-mtd" \
  -F "file=@meeting.wav" \
  -F "response_format=diarized_json" \
  -F "temperature=0"
```

### Output Format

With `response_format=diarized_json`, vLLM returns an OpenAI-compatible
response with structured speaker segments:

```json
{
  "task": "transcribe",
  "duration": 5.1,
  "text": "Let's review the deployment plan. The vLLM server is already running.",
  "segments": [
    {
      "type": "transcript.text.segment",
      "id": "seg_0",
      "start": 0.41,
      "end": 2.8,
      "text": "Let's review the deployment plan.",
      "speaker": "S01"
    },
    {
      "type": "transcript.text.segment",
      "id": "seg_1",
      "start": 3.12,
      "end": 5.1,
      "text": "The vLLM server is already running.",
      "speaker": "S02"
    }
  ],
  "usage": {
    "type": "duration",
    "seconds": 6
  }
}
```

Speaker IDs such as `S01`, `S02`, and `S03` are generated by the model and are
intended to remain consistent within the audio.

Use `response_format=json` if you want the model's native timestamped
transcript instead.

```text
[0.41][S01] Let's review the deployment plan. [2.80]
[3.12][S02] The vLLM server is already running. [5.10]
```

### References

- [Model card](https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize)
- [GitHub repository](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
- [vLLM audio support](https://docs.vllm.ai/en/latest/features/multimodal_inputs.html)