Parameter-free simplified slot attention matches slot attention on Pascal VOC segmentation, indicating competitive assignment dynamics alone explain most clustering

The authors write slot attention as an interacting particle system and derive simplified slot attention (SSA), a parameter-free model obtained by setting the query, key, and value projections to identity and replacing the LayerNorm, GRU, and residual MLP update with weighted-mean aggregation itself. Prior understanding of slot attention depended on its learned components, making it hard to tell how much clustering comes from the attention dynamics themselves; SSA isolates the competitive assignment mechanism by stripping all learnable parameters. The derivation is analytical, simplifying the slot attention update step by step while retaining its column-stochastic assignment weights, so slots keep competing to explain each input feature.

The authors characterise SSA's relation to soft K-means (SKM) and soft spherical K-means (SSKM), and derive USSA, an unnormalised energy-based surrogate whose energy function, in the zero-temperature limit, attains hard 0/1 assignments with clustering geometry governed by dot-product similarity rather than Euclidean distance. SSA and SKM share column-stochastic weights and a weighted-mean update but differ in similarity function; when slots and inputs are restricted to the unit sphere their assignment rules are equivalent, while their updates differ (SSKM projects each updated centroid onto the unit sphere, SSA does not). USSA removes the input normalisation in the weighted mean to isolate the role of the assignment normalisation. The energy function is derived under the column-normalisation constraint via alternating minimisation; the appendix shows the weight update is a softmax and the optimal slot position is a weighted sum, and proves a cluster is better split when the sum of its cross-cluster dot products is predominantly negative.

On Pascal VOC 2012 with frozen DINO features, comparing K-means, SKM, SSKM, USSA, SSA, and full slot attention, SSA is best or second-best on every metric except FG-ARI, closely tracking slot attention, while slot attention's reconstruction MSE is slightly lower. The experiment attributes differences to the segmentation module itself, since all variants share the same encoder, decoder, and training pipeline; SKM's Euclidean similarity tends to produce under-segmented groups, SSKM's spherical projection markedly degrades segmentation, and USSA's weighted sum progressively collapses to a small number of active slots. Results rest on a single dataset, a single frozen encoder (DINO ViT-B/16), and a reconstruction-based training setup, averaged over 3 seeds; FG-ARI ignores background and the authors caution its interpretation, noting the collapsed USSA obtains the highest cluster-based FG-ARI.

The authors offer heuristic interpretations of the omitted components: the GRU's damping resembles a gradient-descent step size, the Q, K, V projections can be read as a learned similarity function stronger than the plain dot product, and LayerNorm relates to spherical K-means and sphere-constrained dynamics, constraining slot growth without fully restricting slots to a unit sphere. These interpretations map the stripped components onto known clustering mechanisms, providing hypotheses for later work to systematically reintroduce components and determine their respective roles in damping, similarity learning, representation refinement, and normalisation. These are heuristic discussions that the authors explicitly state are not tested empirically, and they acknowledge the actual GRU additionally includes a reset gate and nonlinear activations not captured by the comparison.

The result applies to object-centric segmentation with frozen DINO ViT-B/16 features trained on Pascal VOC 2012 with a reconstruction objective, and is meant for researchers and engineers who want to judge where slot attention's clustering comes from or consider replacing learned grouping modules with parameter-free clustering. It enables follow-up work to characterise SSA's fixed points and convergence directly on this dynamics, and to systematically reintroduce the GRU, projections, and LayerNorm to determine their roles in damping, similarity learning, representation refinement, and normalisation; the energy function also opens a probabilistic reading of clustering and exploration of sampling-based alternatives to gradient-based training.

The energy function characterises the surrogate USSA dynamics rather than SSA's own weighted-mean dynamics, leaving a gap between the geometric intuition it provides and the algorithm actually evaluated; SSA's fixed points and convergence properties are not yet characterised directly. Experiments cover a single dataset, a single frozen encoder, and 3 seeds, so the impact of randomness is not quantified, and reconstructing DINO features is a comparatively weak downstream task. The interpretations of the GRU, QKV, and LayerNorm roles are heuristic and not empirically tested. In addition, the metastable clusters and progressive slot deaths observed in USSA dynamics leave open whether they correspond to metastable clusters in self-attention.