Multi-source learning stops guarding every worst-case mix: pulling adversarial weights back to a reference keeps low-rank representations stable across time

Multi-source low-rank estimation can now learn a shared low-rank representation while adversarially reweighting sources under a prior constraint, instead of guarding against the worst mixture over the whole simplex. Group distributionally robust optimization previously optimized against adverse source mixtures over the entire simplex, while low-rank estimation was largely built on pooled or source-averaged objectives; the two had not been combined. In heterogeneous multi-task regression simulations, PILLAR exceeds per-task ridge, truncated SVD, uniformly weighted low-rank gradient descent, row-sparse multi-task regression, nuclear-norm, and maximin benchmarks in worst-source reward.

The method writes the low-rank matrix as a product of two small factors and updates source weights by mirror ascent on the simplex, with a regularizer pulling weights toward the reference mixture. The low-rank part uses Burer-Monteiro factorization with a balancing penalty, and the weight part uses mirror ascent to respect simplex geometry, alternating as bifactored gradient descent with mirror ascent. At the population level the algorithm contracts geometrically from a common initialization across four constant step sizes spanning an eightfold range; in finite samples, error curves match the predicted slope as per-source sample size grows from 100 to 800 and the number of sources from 3 to 12.

The error bound separates low-rank estimation error from the extra cost of adversarial reweighting, with the number of sources entering only logarithmically. Existing group DRO analyses largely target convex parameter spaces or low-dimensional parametric models, where the number of sources often enters polynomially or linearly. In an electronic health record suicide-attempt risk study using calendar periods as sources and age strata as tasks, PILLAR trained on 2005-2008, 2009-2012, and 2013-2016 reaches the highest average transported AUC of 0.784 on 2017-2018, 2019-2020, and 2021-2022, against 0.763 for the best benchmark.

The report is for researchers and engineers handling heterogeneous multi-source data who care about uncertainty in the deployment population's source composition. The method applies when losses are differentiable, the parameter has low-rank structure, and sources are enumerable; simulations cover multi-task regression, and the empirical study covers electronic health record suicide-attempt risk prediction with calendar periods as sources and age strata as tasks. The theory holds when the regularization level meets a stated lower bound, step sizes fall in a given window, sample sizes are sufficiently large, and initialization lies in a local neighborhood.

The theory requires the regularization level to exceed a lower bound, which the authors describe as sufficient for their one-potential proof rather than inherent to the estimator, and they report stable convergence across the whole grid including the unregularized case. The bound's logarithmic dependence on the number of sources relies on the adversarial weights staying sufficiently diffuse; when weights concentrate on a few high-loss sources, the effective sample size saturates. The benefit of prior-informed regularization is largest when the reference distribution is reliable, while weaker regularization is safer under substantial prior misspecification. Robustness is defined through reweighting observed source distributions; robustness to genuinely new environments is observed in simulations and the transport experiment but awaits theoretical characterization.