Fireworks stops shipping generic draft models: adaptive speculative decoding tuned to customer data cuts latency up to 3x
Synopsis
Fireworks has added adaptive speculative execution to FireOptimizer, training the draft model for speculative decoding on a customer's own inference workload instead of generic conversation data. Speculative decoding lets a smaller draft model predict several tokens in parallel for the main model to verify, so the draft's hit rate sets the payoff; generic drafts hit poorly on specialized workloads. In one specialized case a generic draft hit 29% and raised latency 1.5x, while FireOptimizer reached a 76% hit rate and 2x speedup, up to 3x over a generic draft model.
Interpretation
Fireworks launched adaptive speculative execution inside FireOptimizer, which customizes the draft model for speculative decoding to a customer's specific workload rather than generic data. Speculative decoding has typically relied on draft models trained on generic data such as public conversation datasets, and their hit rate drops sharply in specialized settings. In one specialized customer case, the generic draft model's token distribution diverged so far from the target workload that its 29% hit rate made speculative decoding increase latency by 1.5x.
A customized draft model raises the share of draft predictions the main model accepts, turning speculative decoding from a loss into a gain. The speedup from speculative decoding depends on the draft model's hit rate, since a higher hit rate lets the main model keep more correct predictions. On that same case, FireOptimizer reached a 76% hit rate and a 2x speed increase, and Fireworks reports up to 3x speedup over a generic draft model.
Training and evaluation of the draft model are automated, so users only supply data representative of their use case. Developers no longer have to evaluate hit rates or manage hyperparameters, or manually train and benchmark draft models. Data can come from automatically logged production traces or from a sample dataset, the latter useful when data must be anonymized.
Perspective
The demonstrated setting is enterprise reserved deployments, where the feature is offered at no additional cost; availability on on-demand deployments is still being explored. The reported comparison is one specialized case's hit rate and latency plus Cursor's reported roughly 2x reduction in generation latency.
Fireworks states that, like any tuned model, the draft model loses accuracy and needs realignment when production data distributions drift over time. The 3x figure is framed as an upper bound relative to a generic draft model, not a guarantee for every use case.