LLM weights squeezed to a true 1 bit: AF1 hits 61.80 zero-shot on LLaMA-3-70B and cuts memory about 90%

LLM weights can be stored at one bit without extra scaling factors, group parameters or outlier indices inflating the effective width. Earlier binarization-based post-training quantization methods are nominally 1-bit but often land at an effective 2-4 BPW end to end; quantization-aware training yields genuine 1-bit representations but relies on large training data and long optimization. On LLaMA-3-70B, AF1 reaches 61.80 average zero-shot accuracy over seven tasks and 8.91 WikiText2 perplexity at 1.0 effective BPW for target linear weights, above the 3.25-bit HBLLM baseline.

Each linear weight is factored into two binary sign matrices multiplied together, with a learnable continuous scaling vector between them, giving more expressible magnitude structure under the binary constraint. Single-matrix binarization approximates a weight by one scaled binary matrix, whose magnitude matrix is rank one and fits the complex magnitude distributions of LLM weights poorly. In an ablation on LLaMA-3.2-3B, the null-space-aware binary factorization alone lowers WikiText2 perplexity from 8.9e2 to 30.47 and raises seven-task average accuracy to 40.79.

Structural budget is allocated at two levels, across layers and across submodules within a layer, concentrating the limited intermediate dimension where quantization error hurts most. Uniform bit-width configurations ignore how differently layers and submodules respond to binarization error, while local sensitivity estimates alone miss cross-layer error propagation. Adding the two allocation stages on LLaMA-3.2-3B lowers perplexity to 24.21 and raises average accuracy to 43.09; at a 1.00-bit budget, LLaMA-3-8B perplexity falls from 35.84 under uniform allocation to 22.18.

Quantization takes a few GPU hours, and inference memory and bandwidth drop sharply. Quantization-aware training methods reach lower perplexity near 1 bit but need hundreds to thousands of GPU hours, limiting scalability. On LLaMA-2-7B, AF1 uses 5.9 GPU hours to lower perplexity from ARB-LLM's 16.44 to 11.20, about 40x faster than LittleBit and 228x faster than OneBit; at batch size 1 with 1024-token prefill and 4096-token decoding, it saves about 90% memory over BF16 and 68% over GPTQ-W4 on average across three models, with decoding throughput about 2.5x BF16.

Results concern weight-only quantization of target linear weights, evaluated under a 1.0 effective-BPW budget; experiments cover representative LLaMA, Qwen and Gemma models on standard language benchmarks, and efficiency figures come from a GPU implementation at batch size 1 with 1024-token prefill and 4096-token decoding.

The method does not directly compress activations or the KV cache, which can become important in long-context or large-batch serving; the efficiency evaluation rests on a representative GPU implementation with optimized binary matrix multiplication, and integration into broader inference stacks remains a future direction; experiments are limited to LLaMA, Qwen and Gemma on standard language benchmarks, leaving more architectures, broader instruction-tuned variants and real deployment workloads to be validated.