# Retro

Tracing per-query predictive changes through TabICLv2, TabPFN-3, and EXAONE-Tabular, the authors find refinement is highly uneven across depth and often concentrated in later layers; they propose retrospective inference and build Retro, which uses Attention Residuals to adaptively reweight contributions from different depths and query-conditioned gated attention to modulate contextual updates element-wise, shifting refinement earlier and more broadly and reaching top-three overall Elo on TabArena, TALENT, and RelArena while lying on the performance-cost Pareto frontier.

Tracing per-query predictive changes through TabICLv2, TabPFN-3, and EXAONE-Tabular, the authors find refinement is highly uneven across depth and often concentrated in later layers; they propose retrospective inference and build Retro, which uses Attention Residuals to adaptively reweight contributions from different depths and query-conditioned gated attention to modulate contextual updates element-wise, shifting refinement earlier and more broadly and reaching top-three overall Elo on TabArena, TALENT, and RelArena while lying on the performance-cost Pareto frontier.