# Multimodal Federated Learning

In multimodal federated learning, when clients observe only some modalities, aggregating by which modality is available drops encoders that still shape predictions. RASO instead picks each component's contributors by prediction dependence and local-optimizer membership, keeps momentum on encoders, and synchronizes the shared prediction block to its current aggregate. Across KU-HAR, CREMA-D and other datasets under two fusion architectures, concatenation raises KU-HAR macro-F1 from 41.30 under the availability rule to 49.13, while under masked attention RASO leads on UCI-HAR, KU-HAR and CREMA-D and FedOpt leads on CrisisMMD.

In multimodal federated learning, when clients observe only some modalities, aggregating by which modality is available drops encoders that still shape predictions. RASO instead picks each component's contributors by prediction dependence and local-optimizer membership, keeps momentum on encoders, and synchronizes the shared prediction block to its current aggregate. Across KU-HAR, CREMA-D and other datasets under two fusion architectures, concatenation raises KU-HAR macro-F1 from 41.30 under the availability rule to 49.13, while under masked attention RASO leads on UCI-HAR, KU-HAR and CREMA-D and FedOpt leads on CrisisMMD.