Agents stop rewriting only behavior: environment rules become falsifiable hypotheses that get retired when dynamics drift

Agent self-improvement gains a second route: instead of changing model parameters, change what the agent believes about how the environment responds. Prior self-evolving agents mainly improved how they act, through accumulated trajectories, reflection, memory, or distilling experience into reusable skills, which targets inadequate strategy or procedure. On ALFWorld without actor-side transition memory, EvoBelief raises success over the matched Frozen World Model by 23.9% with Claude Haiku 4.5 and 31.3% with GPT-5.4.

Environment knowledge is written as falsifiable hypotheses that must be tested before they may guide a prediction or a plan. A newly formed hypothesis has no authority, and one surprising outcome is not evidence for it and cannot justify a task-directed action as established knowledge. The commitment gate requires support from at least two distinct bindings in at least two distinct episodes, at least one independent of any decision the hypothesis itself influenced, with thresholds fixed before any benchmark run and never tuned on benchmark results.

Counterexamples narrow, demote, or retire rules, so stale knowledge does not keep dragging the agent down when the environment drifts. A rule is first narrowed to the scope still supported by surviving evidence and the commitment condition is recomputed; if evidence no longer suffices it is demoted but keeps its evidence, and it is retired only when repeated counterexamples leave it with no supporting evidence. In every 40-episode MiniGrid stream the key that opens the door switches after episode 20; counterevidence retires the old rule and reopens experimentation, a successor rule is validated, and success returns to its pre-switch level.

Validated rules transfer across tasks because a rule describes the environment rather than a task. Uses inside the task where a rule earned its authority are rare by construction, since validation needs independent evidence from at least two tasks. Across the four ALFWorld configurations, over 80% of the decisions under a validated rule fall in a family other than the one where the rule was validated.

The mechanism is demonstrated in text-based environments, where transitions can be logged and predictions checked when the required outcomes are observable. The intended readers are researchers and engineers working on continual adaptation and scientific discovery with LLM agents, in interactive tasks whose dynamics disagree with the pretrained prior and may change during deployment. The authors list extending it to multimodal, embodied, and real-world software agents as a next step.

The experiments use text-based environments, where transitions can be logged and predictions checked. The authors note that stochastic settings call for statistical tests in place of count thresholds when committing a rule, and that costly or irreversible interventions would run under explicit risk limits or human approval. Where episodic memory already supplies what the prior lacks, it overlaps with explicit beliefs, and learning when to rely on each remains open.