Continually updated models leak when data was used: TMIA reads confidence trajectories to time sample entry and exit, with timestamp F1 above 83%

Continually updated models expose their training history as a sequence of queryable snapshots, adding a temporal dimension to membership privacy: beyond whether a sample was used, one can ask during which period it was used. Earlier membership inference attacks handled only one-directional change, insertion only or removal only, assumed at most one membership transition per sample, and most compared only two consecutive snapshots. On the Pythia-70M and SST-5 update sequence, the adjacent-snapshot UP-MIA placed an entry point at timestamp 40 while the true entry was at timestamp 70, and the per-snapshot TS-MIA placed the entry at timestamp 46.

TMIA does not judge snapshots one at a time; it reads a sample's confidence across the whole snapshot sequence as a time series, locates entry and exit events, and assembles them into membership intervals. The method performs sequence labeling with a sliding-window bidirectional LSTM, whose window restricts attention to local patterns around entry or exit points, then pairs each exit with the nearest preceding unmatched entry in time order so intervals do not overlap and stay correctly ordered. The attacker trains a sequence of update-aligned shadow models on auxiliary data, using 3000 shadow samples to train the attack model and 1200 target samples for evaluation; across 18 model-dataset combinations spanning Pythia-70M, T5, GPT-2, ResNet-18, SimpleCNN and MobileNet, TMIA's timestamp-level F1 exceeds both baselines.

TMIA retains most of its effect even when the shadow models differ from the target in data, architecture or update algorithm. With different shadow and target datasets, timestamp-level F1 drops by at most 4.95%; with different architectures, by at most 2.1%; with different update algorithms, F1 stays at least 88.33%. On LLaMA-3B and LLaMA-8B, using perplexity instead of confidence as the sequence input, TMIA's timestamp-level precision ranges from 95.61% to 99.48% with F1 above 84%.

The evaluation covers three text datasets, SST-5, News20 and MNLI, and three image datasets, CIFAR-10, SVHN and CINIC-10; language models simulate 80 updates and image models 60 updates, each update containing both insertions and removals, while the LLaMA experiments update over 40 timestamps with LoRA fine-tuning. The attack is black-box: the adversary can query snapshots and holds auxiliary shadow data, but does not take part in training and does not know the update algorithm or update contents.

TMIA can reconstruct membership history only within the queried snapshots; insertions or removals occurring entirely between two consecutive queried snapshots are unobservable to any black-box attack. The defense evaluation shows label-only output, dropout and differential privacy all reduce attack effectiveness, but differential privacy gives the strongest protection at the largest loss of model utility, while label-only output yields only about 3% and 11% F1 reductions.