GraphCast takes over only ensemble perturbations: a month of real-observation cycling keeps background errors at least level with a physics model
Synopsis
Placing the machine-learning weather model GraphCast inside a real data-assimilation cycle, with the mean still supplied by the physics-based MPAS model and GraphCast evolving only ensemble perturbations, gives ensemble spread and covariance structure comparable to an all-physics baseline over one month; the EnVar background RMSE against GFS analyses is consistently lower below model level 40, while degradation appears near the model top from interpolation and extrapolation.
Interpretation
A machine-learning weather prediction model can carry the part of data assimilation that evolves ensemble covariances without also producing the mean forecast. Earlier cycling experiments with real observations gave mixed results: some EnKFs diverged with machine-learning forecasts, and others cycled stably but were not compared against physics-based forecasts in the same assimilation system. Cycled for one month (15 April to 14 May 2018) in a two-way coupled EnVar–EnKF system assimilating conventional observations, satellite atmospheric motion vectors, GNSS radio-occultation bending angles and microwave radiances, the GraphCast-evolved ensemble matched the MPAS-evolved baseline in ensemble spread and covariance structure.
The system evolves the mean and the covariances separately, which confines the machine-learning model to propagating perturbations. EnVar estimates background error covariance from the EnKF background ensemble, and the EnKF analysis ensemble is recentered on the EnVar analysis each cycle, so the ensemble forecast model only evolves perturbations around the mean. Baseline configuration C1 uses MPAS for both EnVar and EnKF forecasts, while C2 replaces only the EnKF forecast with GraphCast; all assimilation steps run in MPAS-JEDI on the native MPAS mesh, so C2 interpolates between the two meshes and supplies the soil variables GraphCast lacks.
Below model level 40, the GraphCast configuration's EnVar background has lower RMSE against GFS analyses than the physics baseline, and the improvement grows slowly over the month. The reduction appears at almost all latitudes and model levels, largest over the tropics and Southern Hemisphere and smaller over the Northern Hemisphere. Above level 40, C2 shows pronounced degradation that develops after about two days of cycling and persists all month; independently tuned, both configurations ended with identical ensemble inflation parameters, whereas some earlier machine-learning ensembles needed extra inflation to cycle stably.
Perspective
The result comes from a one-month, six-hourly two-way coupled EnVar–EnKF system assimilating a substantial subset of the global observation network, including conventional observations, satellite atmospheric motion vectors, GNSS radio-occultation bending angles and clear-sky microwave radiances; verification uses GFS analyses as truth, and results above model level 40 are affected by mesh interpolation and the differing model tops.
The degradation above model level 40 has not been traced to a single cause; extrapolation from the mismatched model tops, MPAS's gravity-wave damping layer and reported spurious sensitivities of machine-learning models near the model top may all contribute. Covariance differences in the single-observation experiments also mix in sampling uncertainty from the small ensemble size and can only be read qualitatively, and both systems are under-dispersive.