Skip to main content
Search

# Skipless (Residual-Free) Translator

1 related articles

Subscribe to updates
  1. arXiv

    Two encoders no longer share one space: DuoJEPA adds a single translator layer and lifts chest-radiograph zero-shot AUC from 60.0 to 86.6

    Paired modalities are usually used to pull two encoders into one shared embedding space. DuoJEPA instead keeps a separate LeJEPA self-supervised space per modality and appends to each encoder one residual-free transformer layer that translates its embedding into the other encoder's space, trained by a contrastive loss computed entirely within that target space. On CheXpert Plus chest radiographs and reports it reaches a zero-shot AUC of 86.6 and a cross-modal retrieval score of 78.5, against 60.0 and 64.2 for a reimplementation of LeVLJEPA with the same encoders and data, while a paired-cosine test detects no direct alignment between the two own spaces.

You’re all caught up