Multi-view SAE alignment with pitch transposition as inductive bias recovers orbit structures for chords, keys, and melodic patterns in two music foundation models

The paper argues that many concepts inside a music foundation model are better understood as structured relations rather than isolated features, illustrating this with chords and keys, which are naturally expressed as structured sets such as the 12 transpositions of a chord or the diatonic system within a key. Relative to mainstream interpretability routes such as probing and Sparse Autoencoders that identify individual features with minimal structural assumptions, this work shifts the analytical target from feature identification to structure-based analysis. The claim is supported by the argument in the abstract and by the description of how tonal structures are organized in pitch and time; it is a conceptual argument rather than an independent experimental measurement.

The paper proposes a framework that uses pitch transposition as an inductive bias to induce ordered orbits via multi-view SAE alignment, concretely by generating pitch-shifted input pairs and aligning their SAE representations to discover structured groups of pitch-related features. Compared with identifying isolated features alone, the framework explicitly injects the musical prior of transposition into the representation alignment process so that features are organized into ordered orbits. The method description comes from the abstract, which does not report network scale, training data volume, the form of the alignment loss, or hyperparameters.

Experimental results show that the approach recovers orbit structures corresponding to chords, keys, and melodic patterns across two state-of-the-art music foundation models, while requiring only minimal grounding, for example a few anchor examples, to interpret entire concept families. This offers a structure-level and low-annotation-cost form of interpretation for music foundation models, rather than only per-feature explanation. The evidence comes from experimental results reported in the abstract across two models; the abstract provides no quantitative metrics, baseline comparison numbers, or ablation results.

The framework targets the analysis of internal representations in music foundation models and applies to tonal structures organized in pitch and time, such as chords, keys, and melodic patterns; its setting uses pitch transposition as an inductive bias, generating pitch-shifted input pairs and aligning their SAE representations to induce ordered orbits. The abstract states that the method works across two state-of-the-art music foundation models and that only minimal grounding, such as a few anchor examples, is needed to interpret entire concept families, offering an extendable path toward structure-level, low-annotation-cost interpretation of music models.

The abstract provides no quantitative evaluation metrics, no comparison numbers against baselines such as probing or single-feature SAEs, and no ablations, nor does it specify model scale, data, or the concrete alignment implementation, so the robustness of structure recovery and reproducibility details remain to be confirmed in the full text. Because transposition as an inductive bias is tightly bound to pitch structure, its applicability to musical concepts not dominated by pitch or to other modalities is an open question. In addition, the reading scope here is the abstract only, without figures or body text, so these judgments are limited to what the abstract reports.