Four decades of raw satellite archives handed to multimodal agents: top models finish most analyses but stumble on mapping land change

Land-change analysis is evaluated for the first time on multi-decadal, multi-sensor satellite archives delivered as distributed, so the agent must select data, mask clouds, apply scaling and harmonize across sensors itself. Earlier vision-language and remote-sensing benchmarks ask questions on curated imagery, and Earth-observation agent benchmarks supply preprocessed inputs, short-term tasks or dedicated data-fetching tools, leaving out the raw archives where land-change analysis begins. The benchmark spans 56 sites, 48 core areas of 10–30 km on a side and 8 administrative regions, across six continents and all five Köppen climate groups, with records up to 30 September 2026 and a median of 1,101 Landsat scenes per core area.

Leading agents complete most analyses end to end, and the bottleneck sits in reading change from imagery rather than in writing code over the archive. Six leading models score 92.0% pass3 on questions computed directly from record variables but 61.1% on those requiring a map from imagery; for the same model and task, the odds of a right answer on a mapping question are 0.31 times those on a computed one, 95% interval 0.14–0.66. On the 226 computable and recognition questions, Opus 5.5, GPT-6 Sol, GPT-6.1 Sol and Sonnet 5.5 score 81.0% to 85.4% pass3 with overlapping intervals, and 15 of the 18 questions no model answers right need a map.

The record moves answers from prior knowledge to the pixels but not yet to change on the ground: alter one satellite's data and right answers follow the alteration. In instrument pairs, 53.5% of cases right on the original turn wrong after the alteration, and 91% of changed answers move in the direction of the alteration; placebo pairs that remove 5% of clear scenes change answers in only 12% of cases. All 22 instrument pairs use computable questions; 21 scale one satellite's values from 2018 onward, by up to 20% in 20 pairs and by 5% in one, and one replaces Landsat 8 scenes from 2013–2017 with Landsat 7. Only 3.5% of failed runs attribute the break to the instrument.

The evaluation is limited to 56 sites, 238 questions and records up to 30 September 2026, run in offline CPU-only containers with 4 CPUs and 24 GB of memory, capped at 3,600 s and 300 messages per run. Instrument pairs cover only 22 computable questions, and every alteration is detectable by comparison with a concurrent satellite.

The record helps less on recognition questions, and four models score lower with it than in the closed book; at famous areas the closed book scores 55.9 and the record 50.0. Without the image tool, three leading models change by one to four questions on the 73 recognition questions, not significantly. Some agents abstain heavily on recognition questions that need a map, yet are right in 85% to 98% of the runs where they answer.