# Doubly Robust Estimation

Agent evaluation has relied on fixed benchmarks that report task success or failure but cannot say whether reviewing or steering an agent mid-trajectory is worth it at deployment. Researchers bring micro-randomized trials into agent trajectories, assigning review or direct execution with equal probability at each eligible decision, then estimate causal effects with augmented inverse probability weighting and structural nested mean models. On τ-bench Retail, ToolSandbox and ImpossibleBench, review effects vary substantially across tool-call types and agent states, and the same data yield decision-group risk rankings.

Agent evaluation has relied on fixed benchmarks that report task success or failure but cannot say whether reviewing or steering an agent mid-trajectory is worth it at deployment. Researchers bring micro-randomized trials into agent trajectories, assigning review or direct execution with equal probability at each eligible decision, then estimate causal effects with augmented inverse probability weighting and structural nested mean models. On τ-bench Retail, ToolSandbox and ImpossibleBench, review effects vary substantially across tool-call types and agent states, and the same data yield decision-group risk rankings.

Multimodal health records are often compressed into low-dimensional representations and used as adjustment sets, yet variational autoencoders are trained to reconstruct data rather than to preserve what causal identification needs. In simulations, a representation that discarded a rare code affecting both treatment and mediator gave 95% intervals for the interventional indirect effect with 2% coverage at n = 3,000 under moderate confounding, while reconstruction loss did not reveal the problem; a diagnostic comparing nuisance-function predictive loss flagged it in every replicate.

Multimodal health records are often compressed into low-dimensional representations and used as adjustment sets, yet variational autoencoders are trained to reconstruct data rather than to preserve what causal identification needs. In simulations, a representation that discarded a rare code affecting both treatment and mediator gave 95% intervals for the interventional indirect effect with 2% coverage at n = 3,000 under moderate confounding, while reconstruction loss did not reveal the problem; a diagnostic comparing nuisance-function predictive loss flagged it in every replicate.