Rotating detonation engine mode switching resists control: moving the reference frame with the wave makes deep RL learn reliably
Synopsis
In rotating detonation engines, the detonation wave propagates fast while the operating mode evolves slowly, and that gap makes direct deep reinforcement learning control hard. Reformulating the control problem in a moving reference frame that follows the detonation-wave pattern makes the wave structure appear quasi-steady to the agent and separates the fast and slow processes. In a one-dimensional reduced-order model, controllers trained in the moving frame learn more reliably than those trained in a stationary frame, remain effective over a broader range of actuation periods, and induce rapid transitions between mode-locked states.
Interpretation
Controlling mode transitions in a rotating detonation engine is difficult because detonation propagation and mode evolution sit on two very different timescales, so the target keeps changing quickly for the agent in a stationary frame. Earlier direct application of deep reinforcement learning to such nonlinear dynamics was challenging because of the system's multi-timescale nature, and transitions to oscillatory or chaotic propagation modes can hinder practical operation. In a one-dimensional reduced-order rotating detonation engine model, controllers modulate spatially segmented injection pressure to induce rapid transitions between different mode-locked states across a range of actuation periods, initial states, and target modes.
The approach reformulates the reinforcement learning problem in a moving reference frame that follows the detonation-wave pattern, making the wave structure appear quasi-steady to the agent. This reformulation enables scale separation between fast detonation propagation and slower operating-mode dynamics, instead of asking the agent to handle both in a stationary frame. Across a range of actuation periods, initial states, and target modes, controllers trained in the moving frame learn more reliably than those trained in a stationary frame and remain effective over a broader range of actuation periods.
Perspective
The results come from a one-dimensional reduced-order rotating detonation engine model, with spatially segmented injection pressure as the control input and rapid transitions between mode-locked states as the task. Controllers are evaluated across a range of actuation periods, initial states, and target modes, and the advantage of moving-frame training over stationary-frame training holds within that scope.
Validation is limited to a one-dimensional reduced-order model, and whether the conclusions carry to higher-dimensional or experimental setups is not established. The moving reference frame formulation and scale separation are presented as potentially useful for related multiscale flow-control problems, not as an established general conclusion.