Washington State University

Large language models fine-tuned on proprietary records memorize names and other personal data, and existing privacy-neuron editing inspects only the leaking model, with no PII-free reference to separate identity neurons from useful context. JEPA-Prune trains a public model on surrogate and masked views of the same records, never on the original personal values, and uses a cross-model representation probe, integrated gradients and a scale-invariant specificity score to locate PII-responsive neurons, which are zeroed before deployment without retraining. Across three PII datasets and five models, pruning 1.6% of neurons lowers PII inference accuracy by up to 7.4 and extraction recall by up to 13.9, while unseen-text perplexity stays within 16% of the unpruned model for most LLMs.

Large language models fine-tuned on proprietary records memorize names and other personal data, and existing privacy-neuron editing inspects only the leaking model, with no PII-free reference to separate identity neurons from useful context. JEPA-Prune trains a public model on surrogate and masked views of the same records, never on the original personal values, and uses a cross-model representation probe, integrated gradients and a scale-invariant specificity score to locate PII-responsive neurons, which are zeroed before deployment without retraining. Across three PII datasets and five models, pruning 1.6% of neurons lowers PII inference accuracy by up to 7.4 and extraction recall by up to 13.9, while unseen-text perplexity stays within 16% of the unpruned model for most LLMs.