Matteo Ciferri

MAAS replaces the text-embedding conditioning of a pretrained audio diffusion transformer with region-of-interest fMRI responses, removing the decoder that previously predicted an external stimulus representation such as CLAP or YamNet. On a 7T fMRI dataset it improves acoustic and semantic metrics, including CLAP similarity, pairwise identification accuracy and Mel-SSIM, over baselines such as two-stage fMRI-to-latent conditioning. The framework also uses a fidelity metric grounded in a computational model of auditory cortex to quantify how individual auditory regions contribute to the reconstructed audio.

MAAS replaces the text-embedding conditioning of a pretrained audio diffusion transformer with region-of-interest fMRI responses, removing the decoder that previously predicted an external stimulus representation such as CLAP or YamNet. On a 7T fMRI dataset it improves acoustic and semantic metrics, including CLAP similarity, pairwise identification accuracy and Mel-SSIM, over baselines such as two-stage fMRI-to-latent conditioning. The framework also uses a fidelity metric grounded in a computational model of auditory cortex to quantify how individual auditory regions contribute to the reconstructed audio.