# reflection/Beam-1.0-501B-A23B — vLLM serving recipe

> Reflection's Beam — a sparse MoE language model (501B total / 23B active) with hybrid sliding-window + NoPE full attention, a 1M-token window, Harmony-format reasoning and tool calling, and an optional DFlash2 speculative drafter. Available in NVFP4, FP8 and BF16.

Configuration: hardware `b300` (8× B300) · strategy `single_node_tp` (Tensor Parallel) · variant `default` (NVFP4) · features `reasoning`, `tool_calling`

- Interactive page: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/reflection/Beam-1.0-501B-A23B
- This configuration as JSON: https://vllm-recipes-git-preview-vllm-pr-60762-inferact-inc.vercel.app/reflection/Beam-1.0-501B-A23B.json?hardware=b300

The commands below were rendered for exactly this configuration and take precedence over any example in the guide.

## Other configurations

Change the query string (the same parameters as the page URL) and fetch this URL again. Unknown or incompatible values fall back to the recipe default, so check the Configuration line.

- `hardware`: `h100`, `h200`, `b200`, `gb200`, `b300`, `gb300`
- `variant`: `default`, `fp8`, `bf16`
- `strategy`: `single_node_tp`, `single_node_tep`, `single_node_dep` (multi-node strategies also need `nodes=<n>`)
- `features`: comma-separated list of `reasoning`, `tool_calling`, `spec_decoding` (default: `reasoning`, `tool_calling`)

## Install

Requires vLLM >= 0.33.0.

Docker image: `vllm/vllm-openai:reflection-beam`

## Serve

```bash
export VLLM_ENGINE_READY_TIMEOUT_S=3600
vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 4 \
  --reasoning-parser beam \
  --enable-auto-tool-choice \
  --tool-call-parser beam
```

Docker equivalent:

```bash
docker run --gpus all \
  --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  vllm/vllm-openai:reflection-beam reflection/Beam-1.0-501B-A23B-NVFP4 \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 4 \
  --reasoning-parser beam \
  --enable-auto-tool-choice \
  --tool-call-parser beam
```

## Guide

> Background reference. Commands in it are examples and may target another configuration — use the Serve section above.

### Overview

[Beam](https://huggingface.co/reflection/Beam-1.0-501B-A23B) is Reflection's
sparse mixture-of-experts language model, with 501B total parameters and 23B
active parameters per token. Its 52 layers combine 512-token sliding-window
attention with NoPE full-attention layers, route each token through 8 of 256
experts (layers 0-2 use dense MLPs), and support a context length of
1,048,576 tokens.

Beam uses the Harmony message format, with an `analysis` channel for
reasoning and a `final` channel for the answer. Generation stops on
`<|return|>` at the end of an answer and on `<|call|>` at the end of a tool
call.

### Variants

| Variant | Checkpoint | Weights on disk | Minimum GPU memory | Recommended hardware |
|---|---|---|---|---|
| NVFP4 (default) | [`reflection/Beam-1.0-501B-A23B-NVFP4`](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4) | 294 GB | 353 GB | 4x B200/B300 or one GB200/GB300 tray (TP4); 8x H100 (TP8, Marlin) |
| FP8 | [`reflection/Beam-1.0-501B-A23B-FP8`](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8) | 503 GB | 604 GB | 8x H200 (TP8) |
| BF16 | [`reflection/Beam-1.0-501B-A23B`](https://huggingface.co/reflection/Beam-1.0-501B-A23B) | 1,003 GB | 1,204 GB | 8x B200/B300 (TP8) |

*Minimum GPU memory* is the weights plus 20% headroom for activations,
CUDA graphs and a working KV cache. It is the total across all GPUs, and
it is the figure the variant selector above shows.

The NVFP4 checkpoint quantizes only the routed experts; attention, dense
MLPs, the router, embeddings and the LM head stay in BF16. On Blackwell it
runs W4A4; on Hopper, vLLM serves the FP4 experts through Marlin with BF16
activations.

### Prerequisites

- **vLLM:** the `vllm/vllm-openai:reflection-beam` Docker image. Beam
  support has not shipped in a vLLM release yet.
- **Hardware:** NVIDIA GPUs. Beam's kernels, including a custom all-reduce
  used with expert parallelism, are CUDA-only.
- **Transformers:** the tokenizer uses the Transformers 5 format
  (`TokenizersBackend`).

#### Install vLLM

```bash
docker pull vllm/vllm-openai:reflection-beam
```

The `vllm serve` commands below run inside this image; the command builder
above renders the full `docker run` line.

### Launching the server

#### FP8 with TP8 on H200 or Blackwell

```bash
vllm serve reflection/Beam-1.0-501B-A23B-FP8 \
  --tensor-parallel-size 8 \
  --moe-backend triton \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam
```

This is the best-performing FP8 configuration measured on both H200 and
B300. Set `--moe-backend triton` explicitly; vLLM otherwise selects CUTLASS
for the FP8 MoE. vLLM's default attention backends, FlashAttention 3 on
Hopper and FlashInfer with TRT-LLM kernels on Blackwell, are already the
fastest measured.

#### NVFP4 with TP4 on Blackwell

```bash
vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam
```

At TP4 the weights take about 73 GB per GPU, leaving room for the full
context window many times over; an 8-GPU node can host two replicas. The
first start takes about 10 minutes because FlashInfer autotunes the MoE
kernel. That is close to vLLM's default 600 s engine-ready timeout, so the
generated command sets `VLLM_ENGINE_READY_TIMEOUT_S=3600`.

#### NVFP4 with TP8 on H100

```bash
vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --tensor-parallel-size 8 \
  --moe-backend marlin \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam
```

Set `--moe-backend marlin` explicitly; it is faster than the backend vLLM
selects by default on Hopper. The FP8 checkpoint also fits on 8x H100 but
leaves much less memory for the KV cache.

#### BF16 with TP8 on Blackwell

```bash
vllm serve reflection/Beam-1.0-501B-A23B \
  --tensor-parallel-size 8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam
```

#### Tuning

- **KV cache dtype:** the NVFP4 and FP8 commands use
  `--kv-cache-dtype fp8`, which halves KV memory per token (on 8x H200 with
  the FP8 checkpoint, about 14M to 28M tokens). The BF16 command keeps the
  KV cache in BF16.
- **Parallelism:** tensor parallelism is the recommended single-node layout.
  TEP (`--enable-expert-parallel`) and DEP (`--data-parallel-size 8
  --enable-expert-parallel`) are also supported; DEP is better suited to
  very large batch sizes.

#### MegaMoE with attention DP (NVFP4, Blackwell)

For prefill-heavy, high-concurrency traffic, run attention data-parallel
with expert parallelism on FlashInfer's CuTe DSL NVFP4 MegaMoE kernel,
which overlaps the expert all-to-all with the expert GEMMs. Start from the
NVFP4 command and replace the parallel and MoE settings:

```bash
vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --data-parallel-size 4 \
  --enable-expert-parallel \
  --moe-backend flashinfer_moe_ep_cutedsl \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam
```

- Only for DEP (`--data-parallel-size` with `--enable-expert-parallel`).
  It does not support TEP or pure TP.
- Use `--data-parallel-size 4` on a GB200/GB300 tray or four B200/B300
  GPUs, and `8` across a full 8-GPU node.
- For low-latency serving, stay on TP4 with `flashinfer_trtllm`.

#### DFlash2 speculative decoding

The DFlash2 drafter
([`reflection/Beam-1.0-501B-A23B-DFlash2`](https://huggingface.co/reflection/Beam-1.0-501B-A23B-DFlash2))
shares the target model's embeddings and LM head:

```bash
--speculative-config '{"method":"dflash","model":"reflection/Beam-1.0-501B-A23B-DFlash2","num_speculative_tokens":7,"attention_backend":"FLASH_ATTN","enable_adaptive_verification":true}'
```

With `enable_adaptive_verification`, vLLM predicts how likely each draft
token is to be accepted and verifies only the ones worth the compute,
rather than all 7 every step. It makes little difference at low
concurrency and raises throughput under load. Remove it to verify the
full draft every step.

The drafter supports up to 15 speculative tokens; 7 is recommended. It
speeds up decoding most at low concurrency and adds about 1.7 GB per GPU,
which reduces KV-cache capacity. Acceptance depends on natural-language
input, so benchmark it with a real-text dataset rather than
`--dataset-name random`.

#### Output constraints

By default, the `beam` output grammar constrains every assistant turn from
its first token (reasoning, then an answer or a tool call). Set
`VLLM_BEAM_ALWAYS_CONSTRAIN_OUTPUT=0` to disable it.

### Client usage

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/Beam-1.0-501B-A23B-NVFP4",
    "messages": [{"role": "user", "content": "Hello."}]
  }'
```

Set `model` to the checkpoint being served: `reflection/Beam-1.0-501B-A23B-NVFP4`,
`reflection/Beam-1.0-501B-A23B-FP8`, or `reflection/Beam-1.0-501B-A23B` for
BF16. `/v1/models` lists the served name.

#### Reasoning effort

Beam reasons at the maximum effort, `max` (512), unless the request sets
`reasoning_effort`:

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "reflection/Beam-1.0-501B-A23B-NVFP4",
    "messages": [{"role": "user", "content": "What is 17 * 19?"}],
    "reasoning_effort": "low"
  }'
```

The Rust frontend (`VLLM_USE_RUST_FRONTEND=1`) serves `/v1/chat/completions`
and `/v1/completions`, but not yet `/v1/responses` or the Anthropic-compatible
`/v1/messages` (both return 404). Use the default Python frontend for those
APIs.

### References

- [Beam model card](https://huggingface.co/reflection/Beam-1.0-501B-A23B)
- [Beam FP8](https://huggingface.co/reflection/Beam-1.0-501B-A23B-FP8)
- [Beam NVFP4](https://huggingface.co/reflection/Beam-1.0-501B-A23B-NVFP4)
- [DFlash2 drafter](https://huggingface.co/reflection/Beam-1.0-501B-A23B-DFlash2)