D-SLR matches or beats truncated SVD at three SVDs and cuts prediction changes on LLM embedding tables by about tenfold

Disjointness is lossless under squared error: Proposition 1 shows that for every rank and stored-row count the optimum of the overlapping joint problem is attainable by a disjoint solution, which uses no more parameters and strictly fewer whenever both rows and rank are used. Prior sparse-plus-low-rank models such as Outlier Pursuit and principal component pursuit allow overlapping row or entrywise sparse components and require iterative solvers with tuned regularization weights; removing overlap is shown here not to shrink the attainable optimum, reducing the joint problem to a stored row set plus one SVD of the remaining rows. The result is stated as Proposition 1 with a proof in Appendix A relying on the Eckart–Young theorem and a row-wise error decomposition; the authors note the lossless part is known in slightly different form in the RPCA literature and extend it to every shape while pricing both solutions.

An assumption-free a-posteriori certificate: Corollary 4 combines a spectral floor (Lemma 2) and a row-energy floor (Lemma 3) by taking the larger, Theorem 5 turns this into a score certificate for any returned description, and the block-wise split floor of Theorem 6 shrank the certified gap by an order of magnitude in the reported tests (universal bound median 32.0%, worst 79.1%; split bound median 1.7%, worst 20.5%). Guarantees in low-rank modelling are usually a priori and assume a data model; this certificate is computed after the fit from the given matrix alone, lower-bounds every rank and row count at once, and when positive indicates where a better solution could still exist and by how much. The floors and certificate are supported by Lemmas 2 and 3, Corollary 4, Theorems 5 and 6, and proofs in Appendices B–H; Proposition 8 reduces the split-floor minimization to a sort, and Table 3 reports median and worst bound percentages.

A closed-form, tuning-free algorithm: with a fixed basis each row's rank-r error is its energy outside that basis, so one sort per rank scores every row count, and the grid plus the chosen solution cost about three SVDs with no iterations; the column is filled with the exact tail from the first SVD so the truncated SVD is always a candidate with its true error. Unlike iterative methods that must be re-solved for each penalty weight or each rank-and-sparsity setting, D-SLR traces the whole error-versus-parameters tradeoff in one pass and lets the solution be chosen afterwards by an error target, a parameter limit, or an EBIC-style selection rule. The algorithm is given in Section 7 with defaults, noise estimate and price in Appendix J; Table 2 reports D-SLR within 1% of the best score in 100% of runs at about 2.5 SVDs and 0.45 seconds per matrix over 945 synthetic matrices.

Empirical gains on synthetic and real data: on the synthetic torture test D-SLR was within 1% of the best score in every run (worst ratio 1.01), tuned Outlier Pursuit reached 94% at about 98 SVDs and the convex relaxation 78% at about 946 SVDs; on the embedding tables of Gemma 3 270M and Llama 3.2 1B, at half the table's parameters D-SLR changed about ten times fewer predictions than the truncated SVD and at 90% changed none. These results move the disjoint decomposition from a formal simplification to reproducible compression gains across LLM embedding tables, network traffic and hyperspectral images. The synthetic test has seven cases and 945 matrices, plus a second sweep of 840 matrices over shapes and background ranks; the LLM experiments calibrate on the WikiText-2 train split and evaluate on its test split using held-out cross-entropy and prediction changes; Appendix M reports three further real matrices where D-SLR matched or beat the truncated SVD at most parameter counts.

The result targets compression measured by squared reconstruction error on matrices that are tall relative to their width, where storing a row is cheaper than adding a rank; the authors note D-SLR helps most when some rows fit the shared basis poorly and otherwise returns the truncated SVD. It can serve as a drop-in for the truncated SVD in imaging, scientific dimensionality reduction, neural-network weight and LLM embedding-table compression, and can be combined with further compression such as quantisation. The certificate holds for any price and any valid lower bound, so tighter floors and better partitions plug straight in; weighted inputs with row weights and a column Gram matrix are supported in Appendix O, enabling use with a second-order approximation of cross-entropy for LLM compression.

The certificate is a lower bound rather than an achievable error, and Theorem 5 states it is only an upper bound on the gain and says nothing about whether any description reaches it; the authors also note the certificate loosens when many rows and ranks are affordable, and Appendix M observes the bound fading at the most generous limits. The default pick assumes the leftover error is noise, and Appendix N shows the defaults fail when more than half the rows fit the shared basis poorly. The partitions are heuristics, described in Appendix I, and better partitions remain an open direction. The LLM experiments are a showcase rather than a full compression pipeline, and the weighted objective keeps only the diagonal of the curvature across tokens and treats it as independent of the hidden states, which is an approximation.