Fireworks folds six Surge benchmarks into its Specialized Intelligence Index, testing whether models can get through a workday
Fireworks has added six Surge benchmarks to its Specialized Intelligence Index (SII); the benchmarks are written by practitioners and test whether models can finish real work in messy environments, with ambiguous instructions and calls that require judgment. Earlier evaluations focused more on well-specified tasks such as writing a patch, running a calculation or answering a question. The new benchmarks score the decisions around execution, using expert-authored rubrics that measure correctness, verification, judgment and communication.