DRQ minimizes worst-case reconstruction loss on a fixed quantization grid, improving perplexity and downstream accuracy for LLMs quantized by six PTQ methods without adding inference overhead
The authors show that lower reconstruction loss on calibration data in weight-only PTQ does not guarantee better model performance and can even degrade it on the same calibration data; they propose DRQ, which minimizes worst-case reconstruction loss over a constrained set of input activation distributions by updating only the integer codes of quantized weights while keeping bit width, grouping, scales, and zero points fixed, improving perplexity and average task accuracy across six PTQ methods and both dense and MoE models without adding inference overhead.