Skip to main content

# Exploration in Reinforcement Learning

2 related articles

Subscribe to updates

arXiv Oct 26, 2026
Exploration in physical control stops looking one step ahead: HTC2 scores multi-step futures and tops mean return in eleven of twelve settings
Exploration in physical control often relies on immediate prediction error or single-step physical change, which can miss information that only emerges over extended interaction. HTC2 trains a horizon-conditioned contrastive critic in a learned Hamiltonian phase space, uses the representation distance between a state-action and its same-episode multi-step futures as the primary curiosity signal, and modulates it with bounded multi-step Hamiltonian variation. Combined with SAC and PPO on six DeepMind Control Suite tasks, HTC2 achieves the highest mean final return in eleven of twelve settings.
Exploration in physical control often relies on immediate prediction error or single-step physical change, which can miss information that only emerges over extended interaction. HTC2 trains a horizon-conditioned contrastive critic in a learned Hamiltonian phase space, uses the representation distance between a state-action and its same-episode multi-step futures as the primary curiosity signal, and modulates it with bounded multi-step Hamiltonian variation. Combined with SAC and PPO on six DeepMind Control Suite tasks, HTC2 achieves the highest mean final return in eleven of twelve settings.
Chinese Academy of Sciences Tsinghua University
Shuwei Wang（First author）
Hamiltonian Temporal Contrastive Curiosity Exploration in Reinforcement Learning Hamiltonian World Model Temporal Contrastive Learning
arXiv Oct 2, 2026
Researchers propose a counterfactual-information criterion and build a single simple environment showing that optimal policies for count-based, prediction-error, empowerment, and information-gain intrinsic rewards are Pareto-suboptimal under that criterion
The work proposes a formal exploration criterion that compares policies by the counterfactual information they acquire, meaning how well their histories can substitute for experience under alternative policies, and constructs a single simple environment in which the maximizing policies of specified count-based, prediction-error, empowerment, and information-gain objectives are Pareto-suboptimal at acquiring counterfactual information; it then explains these failures, establishes conditions under which existing intrinsic rewards successfully encourage optimal exploration, and constructs an objective that assigns higher value whenever exploration strictly improves under the criterion.
The work proposes a formal exploration criterion that compares policies by the counterfactual information they acquire, meaning how well their histories can substitute for experience under alternative policies, and constructs a single simple environment in which the maximizing policies of specified count-based, prediction-error, empowerment, and information-gain objectives are Pareto-suboptimal at acquiring counterfactual information; it then explains these failures, establishes conditions under which existing intrinsic rewards successfully encourage optimal exploration, and constructs an objective that assigns higher value whenever exploration strictly improves under the criterion.
Stanford University
Scott W. Viteri（First author）
Intrinsic Rewards Exploration in Reinforcement Learning

You’re all caught up