# FedLore

The work proposes FedLore, which shares a low-rank optimization basis within each round and refreshes it across rounds, enabling exact aggregation in low-rank coordinates, eliminating the identified projection bias, and letting the accumulated model update exceed the per-round rank budget; the authors characterize the aggregation bias and establish an O(T^-1/2) stationarity bound for the projected-SGD variant under a global-gradient coverage condition with standard smoothness and variance assumptions and bounded gradient heterogeneity; on vision and language tasks, including federated pre-training, FedLore outperforms the evaluated low-rank adapter baselines and matches or exceeds full-parameter training while reducing communication and optimizer-state memory.

The work proposes FedLore, which shares a low-rank optimization basis within each round and refreshes it across rounds, enabling exact aggregation in low-rank coordinates, eliminating the identified projection bias, and letting the accumulated model update exceed the per-round rank budget; the authors characterize the aggregation bias and establish an O(T^-1/2) stationarity bound for the projected-SGD variant under a global-gradient coverage condition with standard smoothness and variance assumptions and bounded gradient heterogeneity; on vision and language tasks, including federated pre-training, FedLore outperforms the evaluated low-rank adapter baselines and matches or exceeds full-parameter training while reducing communication and optimizer-state memory.