How much of a policy's value comes from reacting to state: a covariance identity isolates adaptivity, but deterministic policies cannot be audited from their own logs
When a regulator evaluates a learned policy, the inputs and outputs are usually all that is visible, so it is hard to tell whether value comes from acting on the state or from average behavior. This work shows that, in a finite-horizon Markov decision process, the expected-cost difference between a policy and its open-loop marginal equals exactly the covariance between state and action, and it bounds adaptive value by the mutual information the policy uses. Auditing that quantity under bandit feedback needs a sample size inversely proportional to an average KL divergence, while a deterministic non-constant policy is not identifiable from its own logs.