# Centered Kernel Alignment (CKA)

Which target-model hidden layers a speculative-decoding draft model should see can now be chosen by information content and redundancy instead of a fixed positional rule; on Qwen3-4B-Instruct the selected three-layer set averages about 0.085 more accepted tokens than the EAGLE3 default.

Which target-model hidden layers a speculative-decoding draft model should see can now be chosen by information content and redundancy instead of a fixed positional rule; on Qwen3-4B-Instruct the selected three-layer set averages about 0.085 more accepted tokens than the EAGLE3 default.

The work proposes a fusion framework combining representations from three self-supervised objectives, MAE, JEPA, and SimCLR, and a multi-branch architecture with a shared trunk and objective-specific branches; after pretraining on roughly 25,000 spectrogram and CSI samples, at equal output embedding dimension fusion outperforms the evaluated single-objective encoders on all six downstream tasks spanning communication, sensing, and positioning while using about one-third of their parameters, and the multi-branch architecture retains much of the fusion benefit within a single-encoder parameter budget, with representation analyses indicating complementary contributions across objectives and much of the added benefit retained in components orthogonal to MAE's representation subspace.

The work proposes a fusion framework combining representations from three self-supervised objectives, MAE, JEPA, and SimCLR, and a multi-branch architecture with a shared trunk and objective-specific branches; after pretraining on roughly 25,000 spectrogram and CSI samples, at equal output embedding dimension fusion outperforms the evaluated single-objective encoders on all six downstream tasks spanning communication, sensing, and positioning while using about one-third of their parameters, and the multi-branch architecture retains much of the fusion benefit within a single-encoder parameter budget, with representation analyses indicating complementary contributions across objectives and much of the added benefit retained in components orthogonal to MAE's representation subspace.