# GateRQ

Large language models store part of their factual knowledge in hash tables keyed by preceding token sequences, and low-bit quantization erases long-tail facts first, while giving more bits to frequently accessed rows does worse than choosing at random. GateRQ stores each row as a ladder of residual codes, computes each lookup's gate from a 1-bit code, and reads deeper levels only for the lookups the gate ranks highest. On DeepSeek-V4.1-Flash it cuts table memory from 23.6 to 3.04 GiB per GPU and raises vLLM throughput over FP8 tables in GPU memory by about 11% and about 31% on the two workloads, losing about 1 PopQA point at 1.50 bits per lookup.

Large language models store part of their factual knowledge in hash tables keyed by preceding token sequences, and low-bit quantization erases long-tail facts first, while giving more bits to frequently accessed rows does worse than choosing at random. GateRQ stores each row as a ladder of residual codes, computes each lookup's gate from a 1-bit code, and reads deeper levels only for the lookups the gate ranks highest. On DeepSeek-V4.1-Flash it cuts table memory from 23.6 to 3.04 GiB per GPU and raises vLLM throughput over FP8 tables in GPU memory by about 11% and about 31% on the two workloads, losing about 1 PopQA point at 1.50 bits per lookup.