Monitorability rankings can shift with the task mix: pair matching and post-control risk each close a blind spot
OpenAI's alignment researchers add two monitorability metrics beyond g-mean²: one compares models only on task instances both support, reducing ranking bias from different instance mixtures, and one simulates a monitor as a rejection gate with retries, reporting bad-output probability, deployment probability, and the bad-output share among deployed outputs. In synthetic experiments, a known monitorability ordering is recovered by pair matching as capability spread grows.