Trained physics models need no retraining: group averaging enforces symmetry on pretrained predictors, even for non-compact groups
Synopsis
A trained physics surrogate can be made to respect a known symmetry at test time by transforming the input over the symmetry group, transforming the prediction back, and averaging, without changing the architecture or retraining. For compact groups this averaging carries a known non-increasing-risk guarantee and lowered errors on several The Well surrogates; for non-compact groups such as unit rescalings and the Lorentz group, normalized averaging no longer exists, and the Weyl trick and frame averaging gave clear gains mainly out of distribution or in low-data regimes, while models trained directly on invariant features matched or beat them.
Interpretation
For an already trained physics surrogate, transforming the same input over the symmetry group, transforming the prediction back, and averaging yields a predictor that exactly satisfies that symmetry, with no change to architecture or weights. Earlier symmetry approaches usually build the constraint into the network before training, requiring representation-theoretic or invariant-theoretic machinery that is not always compatible with current architectures; post-hoc averaging instead treats the trained model as an arbitrary function. On pretrained surrogates released with The Well, for datasets including active_matter, gray_scott and rayleigh_benard, group-averaged predictions had lower error than the original baseline, and averaging with only one random group element per time step already beat the baseline.
The procedure treats the trained model as an arbitrary function and averages over group orbits: apply a group element to the input, apply the inverse to the output, and average the results. For a compact group this average is the Reynolds operator with respect to the normalized Haar measure, equal to the orthogonal projection onto the equivariant subspace, so under an invariant data distribution and an equivariant target it cannot increase squared risk. Continuous groups are approximated by Monte Carlo sampling, and experiments show improvement even with very few samples; the shear_flow dataset showed no improvement because its initial conditions break the symmetry, indicating the gain depends on the transformations being genuine symmetries of the whole learning problem.
Non-compact groups admit no normalizable Haar measure, so group averaging cannot be carried over directly; the alternatives are the Weyl unitary trick, a data-dependent orthogonal projection, and frame averaging. The Weyl trick complexifies the predictor and group, averages over a compact real form, then restricts back to the real slice; frame averaging averages over a finite set of orbit representatives per input; both enforce symmetry exactly but inherit no compact-group risk guarantee. On the unit-rescaling group, the Weyl projection lowered regression NRMSE and classification KL relative to the baseline when test units were unseen, but increased risk under fixed units; frame averaging prevented catastrophic failures under unit shifts yet substantially reduced accuracy for the fixed-unit-trained regression.
Perspective
The demonstrated scope covers two-dimensional Cartesian-grid surrogates from The Well, synthetic unit-rescaling regression and classification, chaotic dynamics of the driven covariant double well, and Pythia8 quark-gluon jet tagging; the jet data have only partial Lorentz symmetry because the beam axis is fixed, so those averages are tests under selected transformation windows rather than full Lorentz projections.
For non-compact groups the Weyl projection and frame averaging do not consistently beat the baseline, gaining mainly out of distribution or with limited data; the data-dependent projection is least stable at small training-set sizes; and when retraining with symmetry-adapted features is possible, these post-hoc projections match or underperform it.