Skip to main content
Search

# Large Language Model Compression

2 related articles

Subscribe to updates
  1. arXiv

    DictLLM shrinks a 7B model 5.1× with shared dictionaries and sparse coefficients, adding 2.9 points of perplexity

    On mobile devices the binding constraint for large language models is weight memory, not computation. DictLLM splits weight matrices within grouped layers into one shared dictionary plus layer-specific sparse coefficients, all computed from a pretrained model without retraining. On LLaMA-7B it compresses the model 5.1× with only a 2.9-point rise in WikiText-2 perplexity, a compression ratio about 3.6× that of the state-of-the-art low-rank method.

    Columbia University
    Jinho Park（First author）
    Dictionary kernel compressionDictLLMLarge Language Model CompressionNVFP4 Quantization
  2. arXiv

    D-SLR matches or beats truncated SVD at three SVDs and cuts prediction changes on LLM embedding tables by about tenfold

    The authors propose the Disjoint Row-Sparse plus Low-Rank (D-SLR) decomposition, in which each row is either stored verbatim or fitted by a shared low-rank basis but never both; under squared error this restriction is lossless, the algorithm scores every rank and stored-row count in closed form at about three SVDs, and it provides an assumption-free a-posteriori lower-bound certificate; on synthetic matrices and LLM embedding tables D-SLR is never worse than the truncated SVD at equal cost and, at half the table's parameters, changed about ten times fewer model predictions.

    Chalmers University of Technology
    Vincent Szolnoky（First author, Corresponding author）
    D-SLRTruncated SVDSparse plus Low-Rank DecompositionLarge Language Model Compression

You’re all caught up