Mengqi Guo

Mixture-of-experts weight precision carries two separate costs, one for storage and one for decode; DuoRate stores a single 4.27 bpw copy and picks a read prefix per workload from that workload's own prefill, reaching 76.16 average accuracy at a 2.9 bpw read rate on Qwen3-30B-A3B, close to 4-bit formats.

Mixture-of-experts weight precision carries two separate costs, one for storage and one for decode; DuoRate stores a single 4.27 bpw copy and picks a read prefix per workload from that workload's own prefill, reaching 76.16 average accuracy at a 2.9 bpw read rate on Qwen3-30B-A3B, close to 4-bit formats.