MemGuard-Alpha audit finds MIA contamination scores drop to 0.487–0.537 discrimination at a fixed date, and no filtering variant beats the unfiltered ensemble under symmetric transaction costs
The authors introduce MemGuard-Alpha, comprising a composite contamination score combining five membership inference attack (MIA) methods with a temporal proximity feature, plus Cross-Model Memorization Disagreement exploiting variation in training cutoffs across models, and audit both across seven LLMs (124M–7B), 50 S&P 100 constituents, 42,800 prompts and 299,600 prompt-model MIA scores spanning 2019–2024, yielding three negative results: the temporal proximity feature recovers the in-sample label perfectly (ROC-AUC 1.000) because it is a monotone transform of the defining variable, the discriminative power of MIA scores is largely attributable to scale differences between models (raw scores reach AUC up to 0.99 at a fixed date but fall to 0.487–0.