Tingjia Liu

Medical CLIP has mainly aligned high-level semantic features, leaving texture, entity order and attributes poorly captured and limiting fine-grained tasks. Deco-CLIP splits both image and text into two branches: one ViT pretrained by CT-CLIP for semantics, one MAE-pretrained ViT for texture, with a text encoder that forks in its last three layers, shares keys and values, and uses distinct queries, plus threshold-adaptive sample mining and a regularized attention mechanism. On CT-RATE zero-shot tasks, multi-abnormality detection accuracy improves by 10.2% and report-to-volume retrieval recall by 16.2%.

Medical CLIP has mainly aligned high-level semantic features, leaving texture, entity order and attributes poorly captured and limiting fine-grained tasks. Deco-CLIP splits both image and text into two branches: one ViT pretrained by CT-CLIP for semantics, one MAE-pretrained ViT for texture, with a text encoder that forks in its last three layers, shares keys and values, and uses distinct queries, plus threshold-adaptive sample mining and a regularized attention mechanism. On CT-RATE zero-shot tasks, multi-abnormality detection accuracy improves by 10.2% and report-to-volume retrieval recall by 16.2%.