LoRA stops giving every token a fixed rank: noise-floor squelch activates 8.6 of 32 components
SQUELCH treats how many rank-one components a token needs as a detection question, soft-thresholding each token at its own noise level so 8.6 of 32 components activate on average, and beats LoRA by 2.2 to 3.2 points on commonsense reasoning, knowledge, math, code and GLUE.