Skip to main content
Search

# Contrastive Language-Image Pretraining

2 related articles

Subscribe to updates
  1. arXiv

    Federated fine-tuning stops choosing sides: shared adapters plus a local vision projection hit 82.15 harmonic mean across ten datasets

    BLEND splits federated CLIP fine-tuning into shared vision and text adapters plus a per-client vision projection adapter, reaching an average harmonic mean of 82.15 across ten image datasets, 4.81 points above the strongest baseline.

    Zhejiang UniversityUniversity at Buffalo, State University of New York
    Payam Abdisarabshali（First author）
    BLENDFederated LearningVision-Language ModelsAdapter-based Fine-TuningContrastive Language-Image Pretraining
  2. Nature Communications

    Brain maps now decode to tasks, concepts and domains at once, not just a task name

    Functional decoding of brain activation maps has a new route: NiCLIP aligns neuroimaging article text with coordinate-derived activation maps in one shared latent space through contrastive learning, then uses a cognitive ontology to turn similarity into posterior probabilities over tasks, concepts and domains. Widely used correlation decoders and GC-LDA rely on precomputed meta-analytic maps and fixed vocabularies and give task-level predictions only. On Human Connectome Project group-level maps across seven domains, the best configuration reached 62.86% task recall against a 20.71% maximum for Neurosynth and GC-LDA, while the same model dropped markedly on subject-level maps.

    The University of Texas at AustinMcGill UniversityFlorida International UniversityUniversity of Oxford
    Julio A. Peraza（First author）Angela R. Laird（Corresponding author）
    NiCLIPFunctional Decoding of Brain Activation MapsContrastive Language-Image PretrainingCognitive Atlas Ontology

You’re all caught up