Distillation makes secret-keeping models confess more readily, and can suppress animal-preference transfer while preserving math ability
The authors introduce and separately validate two distillation strategies: Distillation for Incrimination (DFI) distills AuditBench secret-keeping models back into their instruction-tuned base, yielding students that confess hidden quirks more often than their teachers, indicating that knowledge of the behavior transfers more readily than the propensity to conceal it; Distillation for Capabilities (DFC) preserves math capability while few-unique-example multi-epoch training and inoculation prompting substantially reduce subliminal transfer of an animal preference used as a misalignment proxy.