DeepSeek R1 and V3 get quantization-aware fine-tuning on Fireworks, keeping FP8 serving without the accuracy drop
Synopsis
Fireworks has added quantization-aware fine-tuning to its FireOptimizer adaptation engine, letting teams tailor the open DeepSeek R1 and V3 models to a specific use case. Previously, LoRA-tuned weights merged and served in FP8 lost noticeable accuracy, and the 671-billion-parameter base alone occupies 671 GB in FP8, above the typical 640 GB of an H100 host. The method fake-quantizes merged weights and activations in the forward pass so LoRA gradients stay in bfloat16 while merged weights follow FP8 numerics; on a French-wine review classification task with 15,000 training and 5,000 evaluation records, QAT tuning beat naive FP8 LoRA tuning.
Interpretation
DeepSeek R1 and V3 can now be fine-tuned with quantization awareness inside Fireworks' FireOptimizer adaptation engine, tuned for quality, latency and cost for a specific use case, and deployed with one click on a dedicated deployment. Fine-tuning these models had been difficult: they are natively trained and served in FP8, while nearly all existing LoRA methods assume unquantized bfloat16 LoRA weights at inference, so training in bfloat16 and serving in FP8 carries a significant accuracy penalty. Fireworks tested the path on a wine-review classification task, filtering French wines from a wine reviews dataset, with 15,000 records for training and 5,000 for evaluation, comparing no fine-tuning, non-QAT fine-tuning and QAT fine-tuning; fine-tuning gave a strong quality improvement and QAT held an edge over naive FP8 LoRA tuning.
During training, merged weights and activations are fake-quantized so the forward pass follows FP8 numerics, simulating the inference quantization setup while training. The difficulty is that LoRA modules are small and must stay in bfloat16 for quality, yet at inference they are merged into the base weights for speed, and converting merged weights from bfloat16 to block-wise FP8 loses precision; QAT keeps LoRA matrix gradients in bfloat16 while the merged weight follows FP8 numerics. The mechanism is shown with a three-line training and deployment recipe for DeepSeek V3, and QAT fine-tuning is stated to be available now for DeepSeek R1 and V3.
Perspective
The demonstrated scope is quantization-aware fine-tuning and one-click dedicated deployment of DeepSeek R1 and V3 on the Fireworks platform, with the quality comparison limited to a French-wine review classification task; QAT support for Llama and Qwen is listed as coming in the next few weeks.
The quality comparison is given on a single classification task without reporting specific metric values; experiments on Llama 3.3 and DeepSeek V2 Lite show FP8 QAT training is superior in accuracy to all training and serving combinations except full bfloat16 for both, so full bfloat16 retains an accuracy edge; QAT support for Llama and Qwen is still to come.