# Low-Rank LLM Compression

Low-rank compression previously assigned a fixed rank during inference, so every token cost the same compute. LRCC adds one lightweight router per Transformer block that selects among a few nested low-rank paths for each token, training only the routers while the low-rank factors stay frozen. Within the same average active-parameter budget, Llama-2-7B gains 7.6 percentage points in average downstream accuracy over static low-rank compression; at matched batch-size-1 decoding latency, Llama-3.2-1B improves both perplexity and downstream accuracy and Llama-2-7B stays competitive, without specialized CUDA kernels.

Low-rank compression previously assigned a fixed rank during inference, so every token cost the same compute. LRCC adds one lightweight router per Transformer block that selects among a few nested low-rank paths for each token, training only the routers while the low-rank factors stay frozen. Within the same average active-parameter budget, Llama-2-7B gains 7.6 percentage points in average downstream accuracy over static low-rank compression; at matched batch-size-1 decoding latency, Llama-3.2-1B improves both perplexity and downstream accuracy and Llama-2-7B stays competitive, without specialized CUDA kernels.

The work introduces QuLoC, a photonic quantum-assisted LLM compression algorithm that uses quantum circuit outputs to gate the retained low-rank components during training and recovers performance through local functional reconstruction followed by end-to-end knowledge distillation; after training the gating coefficients are absorbed into the low-rank factors so the compressed model runs on classical hardware without executing quantum circuits at inference, achieving a 9.73% relative improvement in average accuracy over state-of-the-art baselines on Qwen3.5-4B and comparable or higher average downstream accuracy on LLaMA-7B across different parameter compression ratios.

The work introduces QuLoC, a photonic quantum-assisted LLM compression algorithm that uses quantum circuit outputs to gate the retained low-rank components during training and recovers performance through local functional reconstruction followed by end-to-end knowledge distillation; after training the gating coefficients are absorbed into the low-rank factors so the compressed model runs on classical hardware without executing quantum circuits at inference, achieving a 9.73% relative improvement in average accuracy over state-of-the-art baselines on Qwen3.5-4B and comparable or higher average downstream accuracy on LLaMA-7B across different parameter compression ratios.