Skip to main content
Search

# Residual Stream Activations

2 related articles

Subscribe to updates
  1. OpenAI Alignment 研究与报告

    OpenAI moves safety classifiers inside the model: from GPT-5.6, every assistant reply passes an activation probe

    Starting with GPT-5.6, OpenAI has added a layer of lightweight classifiers, called probes, that read a model's own internal activations on every assistant completion, alongside its external text classifiers. Probes can block a violating turn mid-stream, are more robust to jailbreaks and cheaper to run, and their accuracy rises with the capability of the monitored model; they depend on labeled data, detect only known measurable behavior, and need retraining for each new base model.

    OpenAI
    Activation classifiers (probes)Residual Stream ActivationsMisalignment monitoring
  2. arXiv

    Pre-verdict residual stream activations predict LLM judge position flips with AUROC .621–.850 and transfer to .685–.853

    Using nested grouped cross-validation on 534 JudgeBench pairs and 1,802 MT-Bench comparisons, this work trains L2-regularized linear probes on residual stream activations recorded immediately before the verdict to predict position flips, where swapping candidate order changes the judge's selected response identity; AUROCs reach .621–.850 across four judges, exceeding a combined baseline of verbalized confidence, verdict logits, length differences, and the initial choice by .062–.113, and frozen probes transfer to MT-Bench at .685–.853.

    Stony Brook University
    Hashmath Shaik（First author）
    Linear ProbePosition BiasResidual Stream ActivationsLLM-as-a-JudgeJudgeBench

You’re all caught up