Shuwei Wang

Exploration in physical control often relies on immediate prediction error or single-step physical change, which can miss information that only emerges over extended interaction. HTC2 trains a horizon-conditioned contrastive critic in a learned Hamiltonian phase space, uses the representation distance between a state-action and its same-episode multi-step futures as the primary curiosity signal, and modulates it with bounded multi-step Hamiltonian variation. Combined with SAC and PPO on six DeepMind Control Suite tasks, HTC2 achieves the highest mean final return in eleven of twelve settings.

Exploration in physical control often relies on immediate prediction error or single-step physical change, which can miss information that only emerges over extended interaction. HTC2 trains a horizon-conditioned contrastive critic in a learned Hamiltonian phase space, uses the representation distance between a state-action and its same-episode multi-step futures as the primary curiosity signal, and modulates it with bounded multi-step Hamiltonian variation. Combined with SAC and PPO on six DeepMind Control Suite tasks, HTC2 achieves the highest mean final return in eleven of twelve settings.