# Quantization-Aware Training

3 related articles

Subscribe to updates

Fireworks AI Oct 22, 2026
DeepSeek R1 and V3 get quantization-aware fine-tuning on Fireworks, keeping FP8 serving without the accuracy drop
Fireworks has added quantization-aware fine-tuning to its FireOptimizer adaptation engine, letting teams tailor the open DeepSeek R1 and V3 models to a specific use case. Previously, LoRA-tuned weights merged and served in FP8 lost noticeable accuracy, and the 671-billion-parameter base alone occupies 671 GB in FP8, above the typical 640 GB of an H100 host. The method fake-quantizes merged weights and activations in the forward pass so LoRA gradients stay in bfloat16 while merged weights follow FP8 numerics; on a French-wine review classification task with 15,000 training and 5,000 evaluation records, QAT tuning beat naive FP8 LoRA tuning.
Fireworks has added quantization-aware fine-tuning to its FireOptimizer adaptation engine, letting teams tailor the open DeepSeek R1 and V3 models to a specific use case. Previously, LoRA-tuned weights merged and served in FP8 lost noticeable accuracy, and the 671-billion-parameter base alone occupies 671 GB in FP8, above the typical 640 GB of an H100 host. The method fake-quantizes merged weights and activations in the forward pass so LoRA gradients stay in bfloat16 while merged weights follow FP8 numerics; on a French-wine review classification task with 15,000 training and 5,000 evaluation records, QAT tuning beat naive FP8 LoRA tuning.
Quantization-Aware Training FireOptimizer DeepSeek LoRA Fine-tuning
NVIDIA Technical Blog Oct 17, 2026
NVIDIA Puts Local Coding Agents on One PC: Code, Test and Deploy Without the Cloud Hop
NVIDIA's RTX Spark laptops and compact desktops put local coding-agent inference, testing and deployment on a single machine. The prior workflow was split across a laptop, a cloud model API and a cloud VM. Built on the GB10 system-on-a-chip with up to 128GB of unified memory, NVIDIA reports 4.6x coding-agent performance, 2.7x fine-tuning and 1.8x code compilation versus Intel's Panther Lake 12Xe.
NVIDIA
NVIDIA RTX Spark Nemotron 3.5 Bolt Coding agent Mixture-of-Experts Quantization-Aware Training
arXiv Oct 8, 2026
OnlineQAT uses on-policy distillation on student-generated prefixes to lift Qwen3-1.7B to a 57.28 six-task average at W3A16, 2.90 points above ReasoningQAT
OnlineQAT is a two-stage ultra-low-bit recovery framework that first obtains a usable W2/W3 initialization through block-wise quantization-aware training, then performs on-policy distillation on responses sampled from the quantized student, with a frozen full-precision teacher supplying a sampled reverse-KL signal at each visited prefix; on Qwen3-1.7B it reaches six-task averages of 57.28 at W3A16 and 32.52 at W2A16, improving over ReasoningQAT by 2.90 and 0.44 points, suggesting that student-visited states offer a recovery signal beyond fixed-completion training, particularly at three bits.
OnlineQAT is a two-stage ultra-low-bit recovery framework that first obtains a usable W2/W3 initialization through block-wise quantization-aware training, then performs on-policy distillation on responses sampled from the quantized student, with a frozen full-precision teacher supplying a sampled reverse-KL signal at each visited prefix; on Qwen3-1.7B it reaches six-task averages of 57.28 at W3A16 and 32.52 at W2A16, improving over ReasoningQAT by 2.90 and 0.44 points, suggesting that student-visited states offer a recovery signal beyond fixed-completion training, particularly at three bits.
Hong Kong Polytechnic University Sun Yat-sen University
Wenjun Wang（First author）
OnlineQAT On-Policy Distillation Quantization-Aware Training Qwen3-1.7B Large language models

You’re all caught up

DeepSeek R1 and V3 get quantization-aware fine-tuning on Fireworks, keeping FP8 serving without the accuracy drop

NVIDIA Puts Local Coding Agents on One PC: Code, Test and Deploy Without the Cloud Hop

OnlineQAT uses on-policy distillation on student-generated prefixes to lift Qwen3-1.7B to a 57.28 six-task average at W3A16, 2.90 points above ReasoningQAT