Fireworks ships 3D FireOptimizer: it searches over 100,000 LLM serving combinations to trade the same quality for lower latency
Synopsis
Fireworks has added 3D FireOptimizer to its FireOptimizer tuning stack, a tool that automatically searches configurations spanning model architecture, quantization, speculation, hardware and parallelism. LLM serving must balance speed, throughput cost and quality, and because every knob affects the others the space exceeds 100,000 combinations. The tool prunes that space using production performance sweeps, heuristics and precomputed profiles, then takes a workload's characteristics and SLAs and returns a few candidate configurations; in three case studies, endpoints it produced gave a better QPS-versus-latency tradeoff than the public-platform baseline at the same quality bar.
Interpretation
Fireworks released 3D FireOptimizer, which turns the choice of LLM serving configuration from manual trial and error into an automatic search over a given workload, aiming to hold quality and latency targets while lowering cost. Previously users had to navigate a large set of hardware, low-level optimization and model-family options themselves, and use cases differ widely in prompt length, generation length, prompt repetitiveness, output predictability and request arrival pattern, so no single configuration fits all. Three case studies compare endpoints created with 3D FireOptimizer against the baseline available through the public platform alone, plotting QPS against latency at the same quality bar for whole-codebase agentic code completion with Deepseek V3, code rewriting with Qwen Coder 2.5 14B, and a chatbot on Deepseek V3.
The search covers six coupled dimensions: model architecture and size, parallelism strategy, hardware type, quantization precision, speculation strategy, and the combined effect of these choices. Pulling one lever at a time is not enough, because changing one changes how another should be set; quantizing a model, for example, frees KV cache space, which in turn shifts how the model should be parallelized or replicated and which hardware it can run on. Fireworks draws on its own performance sweeps of varied production workloads across hardware and settings to build heuristics and precomputed profiles that prune the space, then returns a few optimal configurations from the workload characteristics and SLAs supplied.
Perspective
The demonstrated scope is three cases on the Fireworks platform: whole-codebase agentic code completion with Deepseek V3 (120k input, 400k output, 116k tokens cached), code rewriting with Qwen Coder 2.5 14B (2500 prompt tokens, 1500 generation tokens), and a chatbot on Deepseek V3 (5000 input, 500 output, 1500-token prompt cache), compared against the baseline available through the public platform alone.
The three case studies are presented only as curves, with no specific QPS or latency values and no description of how the quality bar is measured; the search depends on Fireworks' own production sweeps and precomputed profiles, and behavior in other environments is not reported.