LinSlot uses the linear representation hypothesis to split slot representations into objects and attributes, beating baselines on DCI and FG-ARI

The paper proposes a three-step generative model: for each factor of each object a concept index and concept value are sampled (with a normal concept prior, making each factor's distribution a Gaussian mixture), each object representation is then written as a linear sum of factor representations, and the image is generated from the object representations. Earlier block-slot attention methods assumed a uniform factorization of object representations into attributes; this work instead grounds the linear combination in the linear representation hypothesis and derives an optimizable ELBO from it. The generative process and joint distribution are given in equations, with the full ELBO derivation in Appendix A.2; the training objective combines a reconstruction term, a slot regularizer and a block regularizer.

Under the generative model and assumptions such as piecewise affine mappings, the paper states an identifiability result: given slots are fixed, blocks are identifiable up to translation and permutation. This gives a theoretical characterization of unsupervised attribute discovery rather than only empirical disentanglement behavior. The proof writes the slot generative model as a linear Gaussian model, invokes the identifiability theorem of Kivva et al., and argues via a mixture-component counting contradiction that the transform must be a block-permutation matrix.

Across four datasets and four slot methods, the cosine similarity between two estimates of the same delta direction is consistently higher than between delta directions of different factors. The authors describe this as the first demonstration of the linear representation hypothesis in slot space, holding across datasets and slot methods including those with pretrained diffusion decoders. Table 1 reports the numbers, e.g. on CLEVR-Easy slot attention gives 0.6610 versus 0.2232 and SLATE+ gives 0.7465 versus 0.1647; on MOVi-C, CODA shows lower absolute values, attributed to over- or under-segmentation in slot inference.

LinSlot and the block-decoder variant LinSlotB outperform all baselines on D, C and FG-ARI on CLEVR-Easy and CLEVR-Hard, match the best method on D and C on CLEVR-Tex, and show lower variance than other baselines. The framework needs neither specialized decoders nor the extensive hyperparameter tuning required by block-slot attention, and improves attribute discovery together with object discovery. Results are presented as DCI and FG-ARI curves; ablations show that removing either two-phase training or mixed-slot decoding substantially affects D and C, removing both leads to training collapse, and the model is insensitive to the GRU and MLP choices.

The framework targets unsupervised static multi-object images and suits settings built on a slot-attention base that need attribute-level disentanglement and interpretable editing; because the slot posterior can be implemented by any slot method and even pretrained, the block discovery module is not tied to a particular slot method, easing substitution of different decoders. The authors list future work as end-to-end identifiability, extension to video, and explicit modeling of global properties and object interactions.

The model does not explicitly represent background or global properties such as lighting and style, which the authors treat as sources of uncertainty. The identifiability result relies on assumptions such as piecewise affine mappings and presumes slots are fixed, leaving end-to-end identifiability open. The specific stable rank and participation ratio values in Table 6 are not listed in the body text, so quantitative conclusions about factor subspace dimensionality rest on the prose description. In addition, the linearity evidence comes from synthetic benchmarks and MOVi-C frames, so generalization to realistic complex scenes remains to be observed.