Deletion requests stop touching most shards: IDSISA concentrates forget sets with influence scores, then distills from untouched shards to recover accuracy
Synopsis
For machine unlearning under privacy rules, IDSISA adds influence-guided shard assignment and cross-shard distillation to SISA's shard-retraining framework. Influence scores are computed once before training and co-locate samples likely to be deleted together, so a deletion request touches fewer shards than under random partitioning; retrained shards also learn from unaffected shards' soft outputs to recover utility lost to isolation. On ResNet-18/CIFAR-10, BiLSTM/IMDB, TCN/IMDB and a fully connected Fashion-MNIST network, the deletion guarantee still comes from retraining itself, and distillation is only a separately quantified utility-recovery step.
Interpretation
A single deletion request has to retrain fewer shards, because samples likely to be deleted together are placed in the same shards before training begins. Random partitioning does not control which shards a deletion touches, so one request often requires retraining most shards. Deleting the 5,000-sample cat class on CIFAR-10 affects 2–3 shards and leaves 7–8 as teachers; under random partitioning a typical request retrains 8–9 shards, while under influence-aware partitioning the typical count is given in different places as 2–4 and as 4–5.
Retrained shards learn the soft outputs of unaffected shards, recovering the global statistical structure that isolated training loses, so forgetting is stronger and retained accuracy higher. SISA's isolated retraining lets each shard see only its own data, losing utility relative to training that could see the full distribution. On ResNet-18/CIFAR-10, residual accuracy on the forgotten class falls from 18.6% to 12.3%, near the 10% random-guess floor, and preservation loss drops from 1.6% to 0.8%; on BiLSTM/IMDB the retained-class change moves from -3.10% to +0.65%.
Cross-shard distillation creates shard-to-shard dependencies that SISA never had, so the deletion guarantee under repeated requests requires cascading retraining over the dependency closure. Single-request certification assumes an empty distillation history; if a shard was taught by another and that teacher's data is later deleted, retraining only the directly affected shards leaves residual dependence. Retraining the reachability closure restores causal exactness to the full aggregated model under any finite deletion sequence; the number of shards retrained equals the closure size, matching the single-request cost in the best case and full SISA in the worst case.
Perspective
The demonstrated scope is a single, isolated deletion request per architecture, covering ResNet-18 on CIFAR-10 and CIFAR-100, BiLSTM on IMDB, TCN on IMDB and a fully connected network on Fashion-MNIST, with all values averaged over 5 random seeds; the cascading mechanism for repeated deletion is measured only on two sequential-request traces and is not part of the main experiments.
The utility recovery from distillation is a separately quantified empirical approximation, and certification covers causal exactness rather than distributional exactness; the theoretical bound relies on convexity assumptions that do not hold strictly for ResNet-18 or BiLSTM, with mean gaps of 0.008–0.013 below the 0.015 heuristic threshold but 99th-percentile tails of 0.019–0.027 above it. On an adversarial sequential trace, 2 of 8 requests collapsed to retraining all shards, so the efficiency advantage holds only in the single-deletion regime.