# Continual Learning

Clinical images arrive in batches by institution, modality and annotated structure, so a segmentation model must absorb a new imaging distribution and new lesion classes without forgetting what it already learned. FGCARL puts bidirectional feature-statistics alignment and Fisher-weighted parameter constraints into one incremental framework that needs no replay of historical patient data. Moving from UCSF-BMSR to BraTS-METS 2023, where domain and classes shift together, FGCARL with Swin UNETR reaches 68.28% mean Dice across the three tumor regions of the new domain, against 60.94% for the strongest comparator, CLAMTS.

Clinical images arrive in batches by institution, modality and annotated structure, so a segmentation model must absorb a new imaging distribution and new lesion classes without forgetting what it already learned. FGCARL puts bidirectional feature-statistics alignment and Fisher-weighted parameter constraints into one incremental framework that needs no replay of historical patient data. Moving from UCSF-BMSR to BraTS-METS 2023, where domain and classes shift together, FGCARL with Swin UNETR reaches 68.28% mean Dice across the three tumor regions of the new domain, against 60.94% for the strongest comparator, CLAMTS.

MEOW converts non-interactive 3D segmentation backbones into open-world interactive segmentation systems without updating any learnable parameters. Interactive segmentation had been confined to the open-set setting, able to predict or reject unknown structures but not to accumulate those predictions. MEOW pairs a frozen feature extractor with parameter-free transductive reasoning and memory consolidation, storing user-confirmed pseudo-labels in memory for later queries to retrieve; on classes never seen in training it keeps improving its own predictions as memory grows, and it transfers zero-shot to micro-CT and other imaging modalities.

MEOW converts non-interactive 3D segmentation backbones into open-world interactive segmentation systems without updating any learnable parameters. Interactive segmentation had been confined to the open-set setting, able to predict or reject unknown structures but not to accumulate those predictions. MEOW pairs a frozen feature extractor with parameter-free transductive reasoning and memory consolidation, storing user-confirmed pseudo-labels in memory for later queries to retrieve; on classes never seen in training it keeps improving its own predictions as memory grows, and it transfers zero-shot to micro-CT and other imaging modalities.

Forgetting in continual learning is usually fought with replay, regularization or architectural changes; STABLE instead works inside the optimizer, treating the distance from the previous task's solution as a budget that tightens the pullback when exceeded and relaxes it once drift returns inside. On the 100-task ImageNet-R stream the same LoRA goes from 0.66% average accuracy with Adam to 73.12%, with average forgetting falling from 60.22% to 10.17%, and CLIP zero-shot transfer stays near its pre-trained level.

Forgetting in continual learning is usually fought with replay, regularization or architectural changes; STABLE instead works inside the optimizer, treating the distance from the previous task's solution as a budget that tightens the pullback when exceeded and relaxes it once drift returns inside. On the 100-task ImageNet-R stream the same LoRA goes from 0.66% average accuracy with Adam to 73.12%, with average forgetting falling from 60.22% to 10.17%, and CLIP zero-shot transfer stays near its pre-trained level.

Mixture-of-experts transformers apply conditional computation only to the MLP, so every token still passes through all attention parameters. NESTOR extends the same routing idea to attention: each token is sent to a few columns, each column has its own attention and experts, and the two levels of sparsity nest hierarchically. In language-model pretraining from 100M to 1B active parameters, the MoE transformer needs 1.13–1.29 times the compute and the dense transformer 3.1–4.1 times the compute to reach the same held-out loss, and this factor shows no downward trend as scale grows.

Mixture-of-experts transformers apply conditional computation only to the MLP, so every token still passes through all attention parameters. NESTOR extends the same routing idea to attention: each token is sent to a few columns, each column has its own attention and experts, and the two levels of sparsity nest hierarchically. In language-model pretraining from 100M to 1B active parameters, the MoE transformer needs 1.13–1.29 times the compute and the dense transformer 3.1–4.1 times the compute to reach the same held-out loss, and this factor shows no downward trend as scale grows.

Progressive neural networks transfer knowledge by keeping every frozen earlier column, so stored state and inference grow with each generation. WeanNet keeps a single frozen parent, feeds its layer-wise activations to a fresh student through a scalar gain, and decays that gain linearly to zero during training, so the parent can be removed at deployment. On a concept-drift task it found the new correct action on drifted states more often than the architecturally matched PNN-Reset control at every tested entropy setting, though it also led on never-drifted states and stayed comparable to training from scratch.

Progressive neural networks transfer knowledge by keeping every frozen earlier column, so stored state and inference grow with each generation. WeanNet keeps a single frozen parent, feeds its layer-wise activations to a fresh student through a scalar gain, and decays that gain linearly to zero during training, so the parent can be removed at deployment. On a concept-drift task it found the new correct action on drifted states more often than the architecturally matched PNN-Reset control at every tested entropy setting, though it also led on never-drifted states and stayed comparable to training from scratch.

Adult endotracheal tube AI models have been hard to apply to newborns, where annotated imaging is scarce and privacy rules block data sharing. An international retrospective study across 30 neonatal intensive care units in 20 countries and territories used continual learning: the model passes from hospital to hospital, and each site retrains it on 50 local radiographs inside its own environment without sharing patient data. The final model averaged 0.67 cm combined endotracheal tube tip and carina localization error, versus 1.29 cm for hospital-specific fine-tuning and 4.30 cm for the adult model used directly, improving localization at 29 of 30 sites and deep-intubation detection at 27 of 30.

Adult endotracheal tube AI models have been hard to apply to newborns, where annotated imaging is scarce and privacy rules block data sharing. An international retrospective study across 30 neonatal intensive care units in 20 countries and territories used continual learning: the model passes from hospital to hospital, and each site retrains it on 50 local radiographs inside its own environment without sharing patient data. The final model averaged 0.67 cm combined endotracheal tube tip and carina localization error, versus 1.29 cm for hospital-specific fine-tuning and 4.30 cm for the adult model used directly, improving localization at 29 of 30 sites and deep-intubation detection at 27 of 30.