# Self-supervised pretraining

Deep learning models for electrocardiograms are mostly trained with supervision tied to specific diagnostic labels, which makes them fragile when patient populations or acquisition settings change. This self-supervised pretraining framework learns features from 918,915 unlabeled 12-lead ECG recordings, with one path explicitly measuring each lead's deviation from normal waveform morphology and another reconstructing cross-lead associations under those deviation cues. With the feature extractor frozen and no downstream fine-tuning, it improves AUROC by up to 6.5% and F1 by up to 12.8% over prior methods on four public datasets.

Deep learning models for electrocardiograms are mostly trained with supervision tied to specific diagnostic labels, which makes them fragile when patient populations or acquisition settings change. This self-supervised pretraining framework learns features from 918,915 unlabeled 12-lead ECG recordings, with one path explicitly measuring each lead's deviation from normal waveform morphology and another reconstructing cross-lead associations under those deviation cues. With the feature extractor frozen and no downstream fine-tuning, it improves AUROC by up to 6.5% and F1 by up to 12.8% over prior methods on four public datasets.

Mouse behavior analysis has relied on self-supervised pose models that encode each animal independently, yet aggression, courtship, mounting and allogrooming are inherently multi-animal. MICE aligns six mouse-pose datasets to a universal 12-keypoint vocabulary and pretrains jointly, pairing a hierarchical individual encoder for per-mouse kinematics with a Perceiver-style social encoder whose learned latent codebook handles any group size, so one checkpoint serves different animal counts without retraining. Across six datasets and four evaluation protocols, MICE matches or exceeds prior keypoint-based baselines, and layer-wise probing shows the social stage improves discriminability beyond the individual encoder.

Mouse behavior analysis has relied on self-supervised pose models that encode each animal independently, yet aggression, courtship, mounting and allogrooming are inherently multi-animal. MICE aligns six mouse-pose datasets to a universal 12-keypoint vocabulary and pretrains jointly, pairing a hierarchical individual encoder for per-mouse kinematics with a Perceiver-style social encoder whose learned latent codebook handles any group size, so one checkpoint serves different animal counts without retraining. Across six datasets and four evaluation protocols, MICE matches or exceeds prior keypoint-based baselines, and layer-wise probing shows the social stage improves discriminability beyond the individual encoder.

Encrypted traffic classification has required retraining the whole encoder whenever application or website labels change, and fine-tuned scores mix in downstream supervised learning. ProtoSPM arranges each bidirectional flow as a matrix of 51 protocol fields and pretrains by masked reconstruction on unlabeled backbone traffic, then freezes the encoder and trains only per-task heads. On ISCX-VPN-app, TLS-120 and CipherSpectrum, the complete 12.03M-parameter model reaches macro-averaged F1 of 61.29%, 67.98% and 71.28%, 2.13 to 26.65 points above a same-input, near-parameter-matched plain field Transformer.

Encrypted traffic classification has required retraining the whole encoder whenever application or website labels change, and fine-tuned scores mix in downstream supervised learning. ProtoSPM arranges each bidirectional flow as a matrix of 51 protocol fields and pretrains by masked reconstruction on unlabeled backbone traffic, then freezes the encoder and trains only per-task heads. On ISCX-VPN-app, TLS-120 and CipherSpectrum, the complete 12.03M-parameter model reaches macro-averaged F1 of 61.29%, 67.98% and 71.28%, 2.13 to 26.65 points above a same-input, near-parameter-matched plain field Transformer.