Skip to main content
Search

# Open-Jev

1 related articles

Subscribe to updates
  1. LMSYS Org

    Decision models skip the explanation: SGLang's Score API reuses a shared query across candidates, so latency barely grows with option count

    For decision tasks such as checking an order's status, an application often needs a category or a score rather than prose. SGLang's Score API lets callers declare label tokens explicitly and read their scores directly, while multi-item scoring (MIS) reuses the repeated query computation inside one request and keeps candidates isolated. On Open-Jev choice tasks, MIS latency stays nearly flat as options grow from 2 to 16: 18.7 ms on Qwen3-0.6B versus 39.6 ms for Generate and 24.3 ms for SIS; the advantage is largest as load rises and is not consistently faster at low load.

You’re all caught up