Reinforcement learning learns Leith coefficients online so coarse 2D turbulence simulations reproduce extreme vorticity events
Synopsis
This work applies scientific multi-agent reinforcement learning (SMARL) to subgrid-scale closure modeling of geophysical turbulence: using the enstrophy spectrum estimated from a few high-fidelity samples as reward, it learns Leith model coefficients online, enabling LES with 160 to 163,840 times coarser resolution than DNS to run stably for simulations about 2000 times the length of the training data and to reproduce DNS kinetic energy spectra and vorticity probability density functions, including the tails that represent extreme events.
Figure 1: Training of SMARL for SGS closures: (1) reference data consists of 5 samples of a short DNS (Eqs. (2a) and (2b)); (2) online training of agents. The state-action map has input state s′(t) (spectrum of enstrophy ˆZ up to LES cutoffwavenumber kc) and domain-averaged output action cl(t) (coefficient in the closure, Eq. (1)) that maximizes the reward r(t). During testing, the policy is coupled to the low-resolution numerical solver to produce a long LES that is 2000× longer than the DNS training set and 1000× the training horizon. Performance for extreme events is evaluated in terms of the vorticity PDF P(ω), against DNS and dynamic physics-based SGS models.
· Page 4Interpretation
It introduces and validates a SMARL online closure-learning framework: agents take the LES enstrophy spectrum as a global state and output Leith coefficients, with reward equal to the inverse of the difference between log DNS and log LES enstrophy spectra, learning the closure without requiring a differentiable solver. SMARL had been applied to 3D homogeneous and wall-bounded turbulence but required known invariants or the law of the wall to define states and actions; this work extends it for the first time to geophysical turbulence prototypes and uses the enstrophy spectrum as state and reward. Training and testing on five 2D turbulence cases, with training using only 5 short DNS snapshots, LES resolution 160 to 163,840 times coarser than DNS, and test simulations about 2000 times the training data length and 1000 times the training horizon.
The RL-Leith closure outperforms dynamic Smagorinsky and dynamic Leith in matching the DNS vorticity probability density function; except for case 4, its PDF matches DNS within uncertainties, including the tails representing extreme events. Traditional dynamic models produce excessive diffusion due to the positive clipping needed for stability, which underestimates extremes; the RL-Leith coefficient distribution is wider and includes negative values, indicating it captures both forward diffusion and backscattering. Comparison against DNS 25-75 quartile uncertainty bands, and Fig. 3 showing RL-Leith predicts interscale enstrophy transfer better than DSmag and DLeith.
Sobol index analysis of the state-action map shows the closure coefficient is most sensitive to low wavenumbers (large scales) of the enstrophy spectrum, next to the high-wavenumber region near the cutoff, with middle wavenumbers having insignificant impact. This provides interpretability evidence for the data-driven closure, mapping the large-scale energy-containing region and the interscale transfer region onto the two main sensitivity zones of the closure coefficient. First-order and total Sobol indices with 95% confidence intervals are reported, along with low-wavenumber kinetic energy fractions of 92.6% for case 1, 94.5% for case 2, 92.3% for case 3, and 93.4% for case 4.
The RL-Leith closure trained on case 1 (Re=20,000) can be used directly on case 5 (Re=300,000, 15 times higher Reynolds number) without new data or retraining, outperforming DSmag and DLeith on kinetic energy spectra and vorticity PDF including tails. Many deep-learning subgrid models generalize poorly across flow regimes and often need transfer learning; this work demonstrates direct generalization to higher Reynolds number and attributes it to the state being the enstrophy spectrum and the spectral similarity of the two cases at the cutoff wavenumber. DNS for case 5 requires 16 times higher spatial resolution than case 1; generalization testing is performed at LES cutoff wavenumber kc=16 and compared against DNS kinetic energy spectra and PDF.
Perspective
The results target weather and ocean modeling scenarios using 2D turbulence prototypes: in a doubly periodic square domain with sinusoidal forcing and, in some cases, a Coriolis force β, the SMARL-learned Leith coefficients let LES run stably at resolutions 160 to 163,840 times coarser than DNS and reproduce statistics. It is intended for researchers and modelers who want closures from a few high-fidelity samples without converting to a differentiable solver, and it provides a starting point for future extension to global climate models and Earth system models.
In case 4, the RL-Leith PDF does not match DNS within uncertainties as in the other cases, suggesting tail characterization still has gaps in some flow regimes. All conclusions come from 2D turbulence prototypes, and extension to 3D geophysical turbulence and global climate models is not tested here. Additionally, this is a fast-parse version with tables and some appendix content not fully presented, so specific case parameter settings and some supplementary analyses cannot be checked item by item here.