MICE, a mouse-behavior foundation model, handles one to four cage mates from a single checkpoint: its social stage sharpens multi-animal behavior
Synopsis
Mouse behavior analysis has relied on self-supervised pose models that encode each animal independently, yet aggression, courtship, mounting and allogrooming are inherently multi-animal. MICE aligns six mouse-pose datasets to a universal 12-keypoint vocabulary and pretrains jointly, pairing a hierarchical individual encoder for per-mouse kinematics with a Perceiver-style social encoder whose learned latent codebook handles any group size, so one checkpoint serves different animal counts without retraining. Across six datasets and four evaluation protocols, MICE matches or exceeds prior keypoint-based baselines, and layer-wise probing shows the social stage improves discriminability beyond the individual encoder.
Interpretation
A single checkpoint now handles behavior recognition for one to four mice in the same arena, instead of training a separate model for each group size. Earlier self-supervised pose models either encoded each animal independently and discarded interaction signal, or hard-coded the animal count at training time, and pretrained on one dataset at a time. In linear probing across six datasets and four evaluation protocols, MICE matches or exceeds BAMS, PointNet, TBERT and hBehaveMAE under both single-dataset and combined pretraining, with the largest gaps on the multi-animal datasets MABe22 and AdaptableSnail.
The model first encodes each mouse's kinematics separately, then lets a set of learned latent codes exchange information across animals to produce an interaction-aware representation. The individual encoder reads single-mouse pose at short, mid and long temporal scales through attention windows that widen linearly with depth; the social encoder uses a Perceiver-style structure in which a fixed number of latent slots read from however many mice are present and write refined codes back to each animal. The six datasets hold roughly 2.2 million training frames, with native skeletons of 7 to 27 keypoints mapped to a universal 12-keypoint vocabulary by synonym matching, missing keypoints zero-padded and masked from dependent features.
Perspective
The demonstrated scope is top-down mouse pose keypoints: six datasets spanning one to four mice per recording and 7 to 27 native keypoints, evaluated under linear probing, unsupervised segmentation, cross-dataset transfer and few-shot label efficiency.
Synonym alignment is manually curated and must be extended by hand for new skeletons; data are mouse-only and filmed from above, with bottom-up and 3D data untested; the pretraining corpus remains small relative to vision or language foundation models, and scaling-law studies are left for future work; behavior label sets are coarse, so ethological subtleties below current annotation resolution will need new benchmarks.