# FlashAttention

When CPUs widen SIMD vectors from 512 to 2048 bits, compute grows but L1 data cache capacity does not, so data shuttles between L1 and L2 and throughput stalls. CacheFlex uses a lightweight ISA extension to convert part of the private L2 into a thread-private scratchpad at runtime, letting software address slots directly and bypass address translation, associative tag lookup, and coherence probes. Modeled in gem5 on Armv8-A SVE, GEMM kernels reach a 1.44x geometric-mean speedup and 40% lower memory-subsystem energy at VL=16; full layers with FlashAttention reach up to 1.78x and 45%, at 2.02% area overhead relative to the L2.

When CPUs widen SIMD vectors from 512 to 2048 bits, compute grows but L1 data cache capacity does not, so data shuttles between L1 and L2 and throughput stalls. CacheFlex uses a lightweight ISA extension to convert part of the private L2 into a thread-private scratchpad at runtime, letting software address slots directly and bypass address translation, associative tag lookup, and coherence probes. Modeled in gem5 on Armv8-A SVE, GEMM kernels reach a 1.44x geometric-mean speedup and 40% lower memory-subsystem energy at VL=16; full layers with FlashAttention reach up to 1.78x and 45%, at 2.02% area overhead relative to the L2.