Skip to main content
Search

# FactorSpec

1 related articles

Subscribe to updates
  1. arXiv

    FactorSpec turns relational constraints into speculative drafts, accepting up to 10.5 tokens per verify step while every record stays valid

    In structured output, relations among a record's fields could determine many tokens, yet serving engines draft from the prompt or a learned head and grammar masks enforce only syntax, so models often violate those relations. FactorSpec compiles a JSON Schema and its relations into a factor graph, treats tokens the constraint uniquely determines as certain drafts through an exact feasibility check, and picks a verify width per request. In SGLang its drafts accept up to 10.5 tokens per verify step on trip planning, up to 1.9 those of the fastest n-gram drafter shape, and decode up to 1.43 faster than speculators tuned per load, at a grammar mask's cost.

You’re all caught up