When can an interpretability claim be trusted: feature dimensionality jumps, while its ridge-regularized version holds one verdict across 12 toy models
Synopsis
Mechanistic interpretability often rests safety claims on thresholded numbers that can flip under small changes to weights, training seed or threshold. Working on toy models and synthetic dictionaries with known ground-truth features, this work characterizes exactly where feature dimensionality is discontinuous and replaces it with a ridge-regularized version whose stability constant grows with feature scale and whose coefficient is optimal. On 12 trained toy models, the regularized sum rule gave the same verdict at all 21 pre-set values of its smoothing parameter, while the version that ignores short feature vectors gave the opposite verdict for two of the models.
Interpretation
Feature dimensionality, a widely used measurement, jumps at specific weight configurations, and the jump is exactly 1, so a claim that a feature exists can be reversed by a small perturbation. The earlier practice applied a norm threshold to feature vectors, and that threshold itself marks small feature vectors as unlearned, producing an apparent failure of a known regularity. The discontinuity set is characterized exactly as the points where a feature's weight vector vanishes while some direction is orthogonal to all other columns; there feature dimensionality and the sum rule both jump by 1, and away from it the local Lipschitz modulus grows like the inverse column norm, an order that cannot be improved.
Replacing feature dimensionality with a regularized version that adds a ridge parameter to the denominator makes the measurement continuous in the weights with explicit stability bounds. The original feature dimensionality had no usable perturbation bound, whereas the regularized version gives a Lipschitz constant independent of matrix shape and proves that constant cannot be made smaller. The ridge leverage score changes by at most a fixed multiple of the operator-norm change of the weights, that constant is attained in the scalar case and by no matrix pair, and the regularized dimensionality's stability constant grows with feature scale with an optimal coefficient and a stated asymptotic additive constant.
Thresholded counts such as the feature recovery rate can be equipped with checkable certificates stating whether a 'feature is recovered' verdict survives a change of dictionary. Previously the recovery rate was only a count that could flip when the threshold or the dictionary changed, with no condition saying whether the verdict persists across dictionaries. On synthetic and trained dictionaries the certificates were never wrong, but across independently trained seeds, and even between two checkpoints of one run, they certify no feature non-trivially.
Perspective
The demonstrated scope is toy models, synthetic dictionaries, and small dictionaries trained with three seeds; real language models were not tested. The certificates held in every test and are informative for small perturbations, but they certify no feature non-trivially across independently trained dictionaries, or even between two checkpoints of one training run.
The optimal scalar value of the additive constant for the regularized dimensionality is proved only on part of the parameter range, with the middle range left as a conjecture and one two-variable inequality flagged as a candidate for a computer-assisted check. The optimal coefficient in the local modulus lies in an interval and is not determined. At the standard threshold, a substantial fraction of features change their recovered status across three seeds.