Skip to main content
Search

# Adaptive Verification

1 related articles

Subscribe to updates
  1. vLLM Blog

    A 501B sparse model gets day-one serving support: coding agents tuned it themselves, reaching 708 tokens per second per request

    Reflection's Beam is a 501B-parameter mixture-of-experts model with 23B active parameters per token, and vLLM supports it from day one with reasoning, tool calling and structured outputs working out of the box. Its first implementation was almost pure PyTorch; coding agents profiled the model, wrote kernels and benchmarked each change, cutting batch-1 decode on TP8 from 8.95 to 3.30 milliseconds per step and an 8×2K prefill from 328 to 176 milliseconds. With its DFlash2 draft head, a single request reaches 708 tokens per second on TP8 at an acceptance length of 3.7.

You’re all caught up