# Diagonal Smoothing

The work introduces OSFP4: by analyzing a multiplicatively dithered FP4 quantizer it obtains a differentiable approximation of the matrix-product quantization error, then jointly optimizes a diagonal smoothing matrix and weight block scales per linear projection, with separate losses for RTN and GPTQ-style SIC rounding; on Llama-3.1-8B-Instruct the W4A4 configuration attains the highest average accuracy among the evaluated methods while retaining roughly 94-97% of vendor NVFP4 prefill throughput.

The work introduces OSFP4: by analyzing a multiplicatively dithered FP4 quantizer it obtains a differentiable approximation of the matrix-product quantization error, then jointly optimizes a diagonal smoothing matrix and weight block scales per linear projection, with separate losses for RTN and GPTQ-style SIC rounding; on Llama-3.1-8B-Instruct the W4A4 configuration attains the highest average accuracy among the evaluated methods while retaining roughly 94-97% of vendor NVFP4 prefill throughput.