Unpaired medical image translation split in two: structure frozen first, OCT-to-OCTA PSNR up to 11.2% higher
Synopsis
Unpaired medical image translation previously packed structure preservation and appearance generation into one source-to-target mapping, which without paired supervision can drop vessels or distort the foveal avascular zone. This work separates the two: a cross-modality shared structure latent space is learned first, then the structure encoder is frozen and a conditional flow model renders only target-modality appearance. On OCT-to-OCTA and brain MR-to-CT, it outperforms all unpaired baselines and, despite no paired supervision, surpasses paired baselines, improving PSNR by up to 11.2% and SSIM by up to 26.4% on OCT-to-OCTA.
Interpretation
Unpaired medical image translation is recast as structure-conditioned target-modality generation, with the anatomy to preserve and the appearance to generate handled in two separate stages. Earlier unpaired methods learned a direct source-to-target mapping through adversarial matching, cycle consistency, contrastive patch learning or diffusion-based conditional generation, without explicitly specifying which anatomical information must stay invariant. On OCT-to-OCTA and brain MR-to-CT, the method outperforms all unpaired baselines and surpasses paired baselines without paired supervision, improving PSNR by up to 11.2% and SSIM by up to 26.4% on OCT-to-OCTA.
Stage one maps source images, target images and structural masks into one shared structure space with a structure encoder-decoder, and the decoder reconstructs multi-class structural maps so the latent representation tracks modality-agnostic anatomy. Source-domain images usually lack reliable structural annotations, and different modalities reveal different structures reliably, for example bone is easier to define in CT than in MR. A structural memory bank stores target-domain masks and their latent representations as prototypes; for each source image it retrieves the most similar target prototypes by Dice similarity and aggregates them into a soft structural target, so source-derived structure latents are supervised using only target-domain annotations.
Stage two freezes the stage-one structure encoder-decoder and trains a target-modality flow model conditioned on its structural representations, initialized from an unconditional target-domain prior, so it only renders appearance under fixed anatomy. The generator no longer has to learn target appearance from scratch or carry structure preservation at the same time. At inference only the source image is given: its structural map is predicted, re-encoded into the shared structure space as the condition, and the target-modality image is produced by solving the conditional flow ODE from Gaussian noise, with no target image, mask or prototype retrieval needed.
Perspective
The demonstrated scope is 2D projection translation from OCT to OCTA and 2D axial slice translation from brain MR to CT; training relies on target-domain structural masks, source-domain coarse masks serve only as optional weak supervision, and cross-center generalization is evaluated on one held-out SynthRAD2023 center.
That held-out-center setting does not cover the full range of institutional, scanner, protocol and population differences; the experiments do not explicitly enforce inter-slice consistency or 3D anatomical topology; mask errors may affect the learned representation; local anatomical hallucination and omission remain possible, and clinical reliability needs broader external validation and expert assessment.