# Masked Image Modeling

4 related articles

Subscribe to updates

arXiv Nov 10, 2026
CroCoMAE moves SAR-optical alignment from whole images to patches, with the largest gains on semantic segmentation
SAR and optical remote sensing images are often treated as largely redundant views, and earlier cross-modal self-supervised methods aligned them only at the whole-image level, leaving fine-grained complementary structure unused. CroCoMAE combines image-level contrastive learning with patch-level masked reconstruction inside a ViT, splitting each patch embedding into a shared subspace aligned across sensors and an unconstrained unshared subspace. On the EuroSAT-SAR scene-classification and DFC2020 semantic-segmentation benchmarks it beats uni-modal self-supervised baselines and retrained cross-modal frameworks in most configurations, with the largest gains on segmentation.
SAR and optical remote sensing images are often treated as largely redundant views, and earlier cross-modal self-supervised methods aligned them only at the whole-image level, leaving fine-grained complementary structure unused. CroCoMAE combines image-level contrastive learning with patch-level masked reconstruction inside a ViT, splitting each patch embedding into a shared subspace aligned across sensors and an unconstrained unshared subspace. On the EuroSAT-SAR scene-classification and DFC2020 semantic-segmentation benchmarks it beats uni-modal self-supervised baselines and retrained cross-modal frameworks in most configurations, with the largest gains on segmentation.
Vito Recchia（First author）
CroCoMAE Cross-Modal Self-Supervised Learning Masked Image Modeling Contrastive Learning SAR-Optical Fusion
bioRxiv Oct 14, 2026
SVC treats each gene's intracellular spread as an image patch: other genes in the same cell supply the context for predicting where unmeasured transcripts sit
Subcellular-resolution spatial transcriptomics can locate transcripts inside cells, but earlier methods targeted single analysis tasks and lacked a unified framework for characterizing and predicting gene localization in a cell-specific, context-aware way. SVC represents each gene's within-cell spatial expression as an image patch and uses other genes in the same cell as context, learning cross-gene spatial dependencies through self-supervised masking. On seqFISH+ mouse embryonic fibroblast data, SVC beat four image-based baselines on all six metrics for predicting subcellular patterns of unmeasured genes, with median gains of 5.2%–21.0% over the more challenging baseline pair.
Subcellular-resolution spatial transcriptomics can locate transcripts inside cells, but earlier methods targeted single analysis tasks and lacked a unified framework for characterizing and predicting gene localization in a cell-specific, context-aware way. SVC represents each gene's within-cell spatial expression as an image patch and uses other genes in the same cell as context, learning cross-gene spatial dependencies through self-supervised masking. On seqFISH+ mouse embryonic fibroblast data, SVC beat four image-based baselines on all six metrics for predicting subcellular patterns of unmeasured genes, with median gains of 5.2%–21.0% over the more challenging baseline pair.
Texas A&M University Yale University
Hui Song Wan（First author）Xiang Zhou（Corresponding author）
SVC Spatial transcriptomics Vision Transformer Masked Image Modeling
arXiv Oct 3, 2026
StreamMAE brings MAE to continuous video streams, reaching 74.0 mIoU on Cityscapes with WT++95h and matching same-data i.i.d. pretraining
The authors build WT++, a 95-hour urban walking-tour video dataset, benchmark MoCo v3, DINO and MAE under a strict sliding-window streaming regime without global shuffling or multi-epoch replay, find that contrastive and self-distillation methods lag while MAE is more robust but still falls short of i.i.d. pretraining, use a controlled experiment to attribute the gap to high intra-batch rather than inter-batch similarity, and propose StreamMAE, which keeps the MAE reconstruction objective while adding stream-aware regularization and motion-biased crop selection, outperforming streaming baselines, matching same-data i.i.d. MAE, and improving as the pretraining stream grows from 12 to 95 hours.
The authors build WT++, a 95-hour urban walking-tour video dataset, benchmark MoCo v3, DINO and MAE under a strict sliding-window streaming regime without global shuffling or multi-epoch replay, find that contrastive and self-distillation methods lag while MAE is more robust but still falls short of i.i.d. pretraining, use a controlled experiment to attribute the gap to high intra-batch rather than inter-batch similarity, and propose StreamMAE, which keeps the MAE reconstruction objective while adding stream-aware regularization and motion-biased crop selection, outperforming streaming baselines, matching same-data i.i.d. MAE, and improving as the pretraining stream grows from 12 to 95 hours.
University of Zagreb University of Technology Nuremberg
Ivan Martinović（First author）
StreamMAE WT++ Dataset Self-Supervised Learning Masked Image Modeling
arXiv Sep 25, 2026
ICHOR self-supervised pretraining on 11,405 ASL CBF scans outperforms structural-MRI pretrained baselines across four downstream tasks
The study introduces ICHOR, a self-supervised pretraining framework for ASL cerebral blood flow (CBF) maps based on 3D masked autoencoders (a ViT-Base encoder with a light decoder), pretrained on 11,405 ASL CBF scans from 14 studies spanning multiple sites and protocols, and evaluated on three diagnostic classification tasks plus one ASL CBF map quality prediction regression task, where it outperformed structural-MRI pretrained baselines BrainIAC, BrainSegFounder, and MedicalNet overall across all four tasks.
The study introduces ICHOR, a self-supervised pretraining framework for ASL cerebral blood flow (CBF) maps based on 3D masked autoencoders (a ViT-Base encoder with a light decoder), pretrained on 11,405 ASL CBF scans from 14 studies spanning multiple sites and protocols, and evaluated on three diagnostic classification tasks plus one ASL CBF map quality prediction regression task, where it outperformed structural-MRI pretrained baselines BrainIAC, BrainSegFounder, and MedicalNet overall across all four tasks.
University of Pennsylvania Philadelphia University University of Maryland, Baltimore
Xavier Beltran-Urbano（First author）Sudipto Dolui（Corresponding author）
ICHOR Masked Image Modeling ASL CBF Maps Vision Transformer LoRA

You’re all caught up

CroCoMAE moves SAR-optical alignment from whole images to patches, with the largest gains on semantic segmentation

SVC treats each gene's intracellular spread as an image patch: other genes in the same cell supply the context for predicting where unmeasured transcripts sit

StreamMAE brings MAE to continuous video streams, reaching 74.0 mIoU on Cityscapes with WT++95h and matching same-data i.i.d. pretraining

ICHOR self-supervised pretraining on 11,405 ASL CBF scans outperforms structural-MRI pretrained baselines across four downstream tasks