Agents stop rewriting behavior: falsifiable environment rules lift reactor-lab completion from 0% to 83%

Self-improvement gains a second route: change what the agent believes about the environment, not how it acts. Earlier self-evolving agents mainly improved how they act from trajectories, reflection, memory, or skills, and did not automatically correct a wrong prior belief about the environment. On ALFWorld without actor-side transition memory, EvoBelief improves success over the matched Frozen World Model, which shares the same rule-blind actor and predictor, by 23.9% with Claude Haiku 4.5 and 31.3% with GPT-5.4.

Environment knowledge is stored as falsifiable hypotheses that are formed, validated, and only then granted authority. A newly inferred rule cannot be used directly; it must first earn independent evidence from later observations or controlled experiments, and is narrowed or retired when counterexamples appear. On MiniGrid the door key switches silently after episode 20; EvoBelief solves most episodes before and after the switch, and after counterevidence retires the old rule it reopens experimentation, validates a successor, and returns to its pre-switch success level.

Validated rules transfer across tasks and directly change predictions and plans. A rule describes the environment rather than a task, so one verified correction can improve many tasks at once, including tasks the agent has never seen. Across the four ALFWorld configurations, over 80% of decisions under a validated rule fall in a family other than the one where the rule was validated; in DiscoveryWorld's Reactor Lab, validated quantitative laws configure the target reactor.

Experiments use text-based environments where transitions can be logged, predictions checked, and the required outcomes observed. Evaluation covers static dynamics in ScienceWorld and ALFWorld, hidden drift in MiniGrid, and the Normal and Challenge tiers of DiscoveryWorld's Reactor Lab and Plant Nutrients themes. The language model stays frozen and all adaptation occurs in the external world model. The authors name extension to multimodal, embodied, and real-world software agents as a next step.

In settings with stochastic effects, committing a rule calls for statistical tests in place of count thresholds; costly or irreversible interventions would run under explicit risk limits or human approval. Where episodic memory already supplies what the prior lacks, it overlaps with explicit beliefs, and learning when to rely on each remains open. On Plant Nutrients the Challenge-tier knowledge score is lower than the Normal tier, and completion and knowledge credit are related but distinct challenges.