Cordial Learning lets agents exchange only low-dimensional outputs and converge to a global optimum on correlated data
The work introduces cordial (correlated and distributed) learning for distributed learning tasks where agents' data are correlated: an agent's label depends on other agents' inputs for the same sample, and those inputs are themselves correlated; the method shares only low-dimensional outputs between agents while training local models to extract informative signals from peers, inducing a game in which each agent's loss depends on others' models, and the authors prove convergence with probability one to a globally optimal solution under a linear model despite the nonconvex global objective, with experiments on structured multi-digit MNIST tasks showing the method remains highly effective even in highly nonlinear settings.