Masoud Kavian

Streaming federated learning trains a shared model from continuously arriving client data, whose streams are often temporally dependent. For clients whose data follow Markov processes, an analysis uses information-theoretic and rate-distortion tools to bound the expected generalization gap of the aggregated model, specializing to linear and self-attention next-token prediction. The bounds show that more communication rounds loosen the guarantee at roughly √(R/N), while temporal dependence enters through the square root of the mixing time; GPT-style pretraining on TinyStories and ETTh2 forecasting both show train and test losses falling with rounds while the gap widens.

Streaming federated learning trains a shared model from continuously arriving client data, whose streams are often temporally dependent. For clients whose data follow Markov processes, an analysis uses information-theoretic and rate-distortion tools to bound the expected generalization gap of the aggregated model, specializing to linear and self-attention next-token prediction. The bounds show that more communication rounds loosen the guarantee at roughly √(R/N), while temporal dependence enters through the square root of the mixing time; GPT-style pretraining on TinyStories and ETTh2 forecasting both show train and test losses falling with rounds while the gap widens.