Personalized federated learning now tracks per-sample representation drift: 63.05% on CIFAR-100, 5.96 points above the strongest baseline

Personalized federated learning gains a way to tune historical supervision per sample: samples within one client no longer receive the same historical constraint, and samples whose representations change more receive stronger historical distillation. Earlier personalization methods characterized clients mainly through data distributions, model components or inter-client relationships, and historical supervision was typically built at the model or client level, leaving sample-level representation change across rounds largely unused. Under the Practical setting on CIFAR-100 with α = 0.1, FedEAGD reaches 63.05% accuracy, 5.96 percentage points above the strongest evaluated baseline; on CIFAR-10 in the same setting it reaches 92.27%, 1.54 points higher.

Each client keeps an exponential moving average copy of its shared feature extractor as a historical teacher, which takes no gradient updates and serves only as a historical representation reference. The exponential moving average teacher compresses the client's retained shared-model states across rounds into one exponentially time-weighted summary, without storing the full sequence of historical models. For each sample, the cosine similarity between the current shared representation and the teacher representation gives the drift, and the distillation coefficient is the base coefficient times one plus drift, clipped at a preset upper bound, so stronger drift yields stronger but bounded supervision.

Personalized information enters the shared representation gradually through a gated residual, with the fusion coefficient at zero before an activation round, rising linearly to one over a preset ramp-up period, then staying at one. Applying the personalized residual from the outset could interfere with early shared-representation learning, so fusion is released in stages by communication round. Ablations on CIFAR-100 under the Pathological setting and CIFAR-10 under the Practical setting show that removing the historical teacher costs 1.26 and 1.09 percentage points, replacing adaptive distillation weighting with a fixed 0.50 costs 0.74 and 0.64 points, and disabling progressive gated residual fusion costs 0.51 and 0.43 points.

The demonstrated scope is image classification on MNIST, CIFAR-10, CIFAR-100 and DomainNet, by default with 20 clients, 100 communication rounds and full participation, using a small backbone of two convolutional blocks with squeeze-and-excitation modules; DomainNet uses six clients corresponding to its six visual domains. Communication is counted as the shared feature extractor transferred in both directions per participating client per round; on CIFAR-100 that extractor holds 2,153,216 parameters, about 8.61 MB in 32-bit floating point and about 17.23 MB bidirectional, excluding protocol overhead and not a measurement of network latency or total system-wide traffic. The authors propose future work on asynchronous client participation, deployment under communication constraints, more complex heterogeneous network environments, and distributed systems such as edge computing and the Internet of Things.

The authors report that lowering client participation hurts CIFAR-100 more than CIFAR-10: going from participation 1.0 to 0.2 drops CIFAR-10 from 79.78% to 76.95% and CIFAR-100 from 45.74% to 35.50%. Raising the client count from 20 to 50 and 100 lowers CIFAR-100 accuracy from 63.05% to 59.22% and 50.73%, which the authors associate with more fragmented local data. On MNIST most personalized methods already exceed 99%, so differences are small, and on DomainNet the margin over FedAS is only 0.40 percentage points. The communication estimate excludes protocol overhead, and the authors state it should not be read as a measurement of network latency or total system-wide traffic.