Public articles linked to the same research event.
arXiv The authors propose AdaStep, which casts how strongly a group-derived local advantage should modify the trajectory-level signal as a mean-squared-error estimation problem for the latent step advantage and, under an explicit conditional sampling assumption, derives an optimal per-state shrinkage coefficient with a signal-to-total-variance interpretation: it preserves local credit when return variation is attributable to the selected action and suppresses it when downstream randomness dominates; the method needs only lightweight scalar computation, with no critic, additional rollouts, or extra model inference, and improves consistently over baselines across three model backbones on ALFWorld, WebShop, and ScienceWorld at low computational cost.
The authors propose AdaStep, which casts how strongly a group-derived local advantage should modify the trajectory-level signal as a mean-squared-error estimation problem for the latent step advantage and, under an explicit conditional sampling assumption, derives an optimal per-state shrinkage coefficient with a signal-to-total-variance interpretation: it preserves local credit when return variation is attributable to the selected action and suppresses it when downstream randomness dominates; the method needs only lightweight scalar computation, with no critic, additional rollouts, or extra model inference, and improves consistently over baselines across three model backbones on ALFWorld, WebShop, and ScienceWorld at low computational cost.
The authors propose AdaStep, which casts how strongly a group-derived local advantage should modify the trajectory-level signal as a mean-squared-error estimation problem for the latent step advantage and, under an explicit conditional sampling assumption, derives an optimal per-state shrinkage coefficient with a signal-to-total-variance interpretation: it preserves local credit when return variation is attributable to the selected action and suppresses it when downstream randomness dominates; the method needs only lightweight scalar computation, with no critic, additional rollouts, or extra model inference, and improves consistently over baselines across three model backbones on ALFWorld, WebShop, and ScienceWorld at low computational cost.
The authors propose AdaStep, which casts how strongly a group-derived local advantage should modify the trajectory-level signal as a mean-squared-error estimation problem for the latent step advantage and, under an explicit conditional sampling assumption, derives an optimal per-state shrinkage coefficient with a signal-to-total-variance interpretation: it preserves local credit when return variation is attributable to the selected action and suppresses it when downstream randomness dominates; the method needs only lightweight scalar computation, with no critic, additional rollouts, or extra model inference, and improves consistently over baselines across three model backbones on ALFWorld, WebShop, and ScienceWorld at low computational cost.