OpenAI moves safety classifiers inside the model: from GPT-5.6, every assistant reply passes an activation probe

OpenAI has added a safety classifier that reads the model's internal activations directly, on top of its existing text classifiers, and runs it on internal and external traffic of its most capable models. The earlier stack relied mainly on classifiers that run on user prompts and assistant completion text, seeing only the words the model produces. The layer was added starting from GPT-5.6 as one component of an internal misalignment monitoring system, alongside model-based review, automated investigation and human response.

A probe takes a residual-stream activation vector, typically from a layer around the middle of the model, and outputs a binary-classification logit, producing one score per sampled token. The architecture builds on Attention probes with an MLP plus QV-attention and an exponential moving average, cutting streaming state from memory that grows with window size to memory that grows only with the number of heads. In robustness tests with benign prefixes before a violative suffix, the MultiMax probe was significantly more affected than Max of Rolling Means, which behaved much like the exponentially decaying attention; softmax aggregation of prefix scores modestly improved the recall/flag-rate Pareto front over a hard max.

Probes run in a streaming fashion on the inference engine with only a minimal token-pooling buffer, so their system reliability is close to the model's own and they are easier to make fail-closed. An external text classifier runs as a separate service with a different failure mode from the model. When a logit crosses a threshold, streaming to the user stops and the message is escalated to a reasoning model for final judgement, and tool calls are blocked until marked safe.

Probe accuracy rises with the capability of the monitored model, transfers well across post-training stages, and transfers more limitedly across pre-training checkpoints. A text classifier sees the same input on a fixed task and therefore performs the same, whereas a probe's input changes with the model. OpenAI reports probe performance continuing to increase on its frontier models, with excellent transfer across many post-training stages and more limited transfer across pre-training checkpoints.

The report covers OpenAI's description of the probe component of its internal misalignment monitoring system, including architecture, training objective, deployment and several internal experiments; the text states the figures come from various internal experiments and are not necessarily representative of deployed classifiers.

Probes rely on labeled training data and can only detect known, measurable kinds of undesirable activity; they are tightly coupled to the monitored model's forward pass, so a probe trained on one model's activations is incompatible with a model of different weight initialization and must be retrained for each new base model, and performance can degrade as weights diverge from the training checkpoint, motivating periodic retraining. The text also notes that real-world attacks can adapt to signals from the safety stack, so extensive red-teaming remains part of the process.