# Group-Equivariant Vision Transformer

The authors propose REViT-v2, which restricts group-convolutional self-attention to local windows (wG-CSA) and builds a multi-stage equivariant feature pyramid from group-convolutional tokenization and downsampling, reducing the spatial complexity of roto-reflection equivariant attention from quadratic to linear in image area; on ImageNet-1K the equivariant REViT-v2-S reaches 79.27% and 80.9% Top-1, above rotation/flip-augmented ViT-S (72.08%) and RE-ResNet (77.37%), and on Rotated MNIST it keeps about 98.26% accuracy with smaller model size, FLOPs, latency, and peak memory.

The authors propose REViT-v2, which restricts group-convolutional self-attention to local windows (wG-CSA) and builds a multi-stage equivariant feature pyramid from group-convolutional tokenization and downsampling, reducing the spatial complexity of roto-reflection equivariant attention from quadratic to linear in image area; on ImageNet-1K the equivariant REViT-v2-S reaches 79.27% and 80.9% Top-1, above rotation/flip-augmented ViT-S (72.08%) and RE-ResNet (77.37%), and on Rotated MNIST it keeps about 98.26% accuracy with smaller model size, FLOPs, latency, and peak memory.