# FlyWheel

FlyWheel is a set of TPU-native attention kernels covering both softmax attention and Gated DeltaNet linear attention. It targets matrix units left idle by repeated operand loads, vector-heavy work and dependency stalls, by coordinating data layouts, recasting triangular inversion and convolution as matrix products, and explicitly interleaving independent computation. On TPU v6e, softmax attention runs up to 3.49x faster than RPA-v3 and up to 1.56x faster than Splash Attention, reaching 81% of peak BF16 throughput.

FlyWheel is a set of TPU-native attention kernels covering both softmax attention and Gated DeltaNet linear attention. It targets matrix units left idle by repeated operand loads, vector-heavy work and dependency stalls, by coordinating data layouts, recasting triangular inversion and convolution as matrix products, and explicitly interleaving independent computation. On TPU v6e, softmax attention runs up to 3.49x faster than RPA-v3 and up to 1.56x faster than Splash Attention, reaching 81% of peak BF16 throughput.