Two independently trained models stop flattening each other: constraining only task-score increments lifts six language tasks by 0.52 points on average
Independently trained models hold complementary capabilities, yet knowledge transfer usually shrinks differences in predictions or representations, which erodes that complementarity. Corroborative Learning has each model learn only an increment to its task scores, then reconstructs those increments from the peer's representations on held-out data groups, constraining directions that reconstruct poorly more strongly. Across six language tasks, 24 models and 36 pairs, it improves on capacity-matched independent adaptation on all six tasks, with a mean gain of 0.52 percentage points, while both models still deploy independently at inference.