# Vision Transformers

Blockwise distillation architecture search picks child networks by summing each replacement block's isolated score, an additive assumption rarely tested directly. On DeiT-S and DeiT-B, as the block-latency budget drops from 60% to 55%, the additive integer linear program begins to trail a pairwise integer quadratic program that models block interactions, by 3.3–16.6 top-1 points on ImageNet-100 after fine-tuning. The error splits into a metric part from the score's nonlinearity and a functional interaction part, dominant on DeiT-S and DeiT-B respectively; a hybrid search that ranks with single-block logits and measures only a few candidates needs no dedicated pair measurements and beats the pairwise search in 13 of 14 full ImageNet-1k settings.

Blockwise distillation architecture search picks child networks by summing each replacement block's isolated score, an additive assumption rarely tested directly. On DeiT-S and DeiT-B, as the block-latency budget drops from 60% to 55%, the additive integer linear program begins to trail a pairwise integer quadratic program that models block interactions, by 3.3–16.6 top-1 points on ImageNet-100 after fine-tuning. The error splits into a metric part from the score's nonlinearity and a functional interaction part, dominant on DeiT-S and DeiT-B respectively; a hybrid search that ranks with single-block logits and measures only a few candidates needs no dedicated pair measurements and beats the pairwise search in 13 of 14 full ImageNet-1k settings.

A cross-architecture comparison of CNNs, ViTs and Vision Mamba uses human eye-tracking heatmaps as an independent benchmark to quantify how closely model saliency matches human gaze, finding that Mamba reaches the highest localization scores on chest X-rays while its saliency intensity is nearly uncorrelated with human fixations.

A cross-architecture comparison of CNNs, ViTs and Vision Mamba uses human eye-tracking heatmaps as an independent benchmark to quantify how closely model saliency matches human gaze, finding that Mamba reaches the highest localization scores on chest X-rays while its saliency intensity is nearly uncorrelated with human fixations.

Medical transfer learning has long inherited spatial-attention backbones pretrained on natural images, yet layout priors do not survive shifts across scanners and modalities. FAD-ViT moves global modeling from spatial tokens to channel tokens and lets depthwise convolution handle local spatial extraction alone, strictly decoupling the two axes. Across 23 classification, segmentation and low-data tasks, the two axes are nearly tied from scratch, while identical ImageNet pretraining yields up to roughly three times the transfer gain on the feature axis, raising ultrasound segmentation Dice from 53.1 to 77.7 at about a quarter of the comparison's parameters.

Medical transfer learning has long inherited spatial-attention backbones pretrained on natural images, yet layout priors do not survive shifts across scanners and modalities. FAD-ViT moves global modeling from spatial tokens to channel tokens and lets depthwise convolution handle local spatial extraction alone, strictly decoupling the two axes. Across 23 classification, segmentation and low-data tasks, the two axes are nearly tied from scratch, while identical ImageNet pretraining yields up to roughly three times the transfer gain on the feature axis, raising ultrasound segmentation Dice from 53.1 to 77.7 at about a quarter of the comparison's parameters.