Victor Villin

Inverse reinforcement learning aims to recover expert rewards that transfer across environments with differing dynamics, and expert demonstrations are costly, so choosing which dynamics to collect them under matters. A two-phase approach first cycles through the available dynamics to obtain a reliable reward estimate, then actively selects dynamics to reduce uncertainty about the expert reward, yielding sample-complexity guarantees on transfer that hold for any dynamics with a rate independent of the number of available dynamics. Empirically, the method significantly reduces the number of expert demonstrations needed to learn a transferable reward.

Inverse reinforcement learning aims to recover expert rewards that transfer across environments with differing dynamics, and expert demonstrations are costly, so choosing which dynamics to collect them under matters. A two-phase approach first cycles through the available dynamics to obtain a reliable reward estimate, then actively selects dynamics to reduce uncertainty about the expert reward, yielding sample-complexity guarantees on transfer that hold for any dynamics with a rate independent of the number of available dynamics. Empirically, the method significantly reduces the number of expert demonstrations needed to learn a transferable reward.