# Linear Probing

DNA-LLM2Vec converts pretrained causal DNA decoders into frozen bidirectional encoders in stages: it opens bidirectional context, repairs the resulting mismatch with masked next-token prediction, then adds contrastive views encoding strand and positional invariance. Across two backbones, DNAGPT and HyenaDNA, and seven biological task families, the benefit is not uniform but depends on task structure and backbone; crop contrastive learning helps most for extended-context families such as enhancer–promoter interaction and Virus-COVID, reaching frozen Avg-EPI MCC of 27.5% and 26.7%.

DNA-LLM2Vec converts pretrained causal DNA decoders into frozen bidirectional encoders in stages: it opens bidirectional context, repairs the resulting mismatch with masked next-token prediction, then adds contrastive views encoding strand and positional invariance. Across two backbones, DNAGPT and HyenaDNA, and seven biological task families, the benefit is not uniform but depends on task structure and backbone; crop contrastive learning helps most for extended-context families such as enhancer–promoter interaction and Virus-COVID, reaching frozen Avg-EPI MCC of 27.5% and 26.7%.

DisParQ learns spatially grounded, discrete concept representations from a powerful frozen vision-only self-supervised backbone without class labels or language supervision: each image patch is assigned to exactly one concept from a learnable prototype dictionary, only a sparse subset of concepts may activate per image, and continuous residuals quantized into discrete attributes capture how each concept varies; a spatial decoder reconstructs the backbone's representation from concepts and attributes alone. Across seven datasets it reaches 83.2% top-1 on ImageNet linear probing, closely matching its frozen DINOv2 teacher, achieves higher concept consistency than language-aligned models, remains competitive on fine-grained recognition, and enables cross-category part-based retrieval.

DisParQ learns spatially grounded, discrete concept representations from a powerful frozen vision-only self-supervised backbone without class labels or language supervision: each image patch is assigned to exactly one concept from a learnable prototype dictionary, only a sparse subset of concepts may activate per image, and continuous residuals quantized into discrete attributes capture how each concept varies; a spatial decoder reconstructs the backbone's representation from concepts and attributes alone. Across seven datasets it reaches 83.2% top-1 on ImageNet linear probing, closely matching its frozen DINOv2 teacher, achieves higher concept consistency than language-aligned models, remains competitive on fine-grained recognition, and enables cross-category part-based retrieval.