# FMHA attention kernels

PyTorch users on AMD MI350-series GPUs can now reach FlyDSL kernels through existing operator APIs for dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm, TopK, and attention via AOTriton. Those operators previously ran on ATen, Triton, or AITER ASM implementations. On MI355X operator suites, FlyDSL gives a 1.10x geometric-mean speedup over the faster ATen/Triton baseline at each of 15 BF16 dense-GEMM shapes, and 1.58x and 1.68x over ATen for MXFP8 and MXFP4. End-to-end vLLM whole-request speedups reach 1.13x for BF16, 2.04x for MXFP8, and 3.01x for MXFP4, depending on model and concurrency.

PyTorch users on AMD MI350-series GPUs can now reach FlyDSL kernels through existing operator APIs for dense and grouped GEMM, MXFP8/MXFP4 scaled GEMM, RMSNorm, TopK, and attention via AOTriton. Those operators previously ran on ATen, Triton, or AITER ASM implementations. On MI355X operator suites, FlyDSL gives a 1.10x geometric-mean speedup over the faster ATen/Triton baseline at each of 15 BF16 dense-GEMM shapes, and 1.58x and 1.68x over ATen for MXFP8 and MXFP4. End-to-end vLLM whole-request speedups reach 1.13x for BF16, 2.04x for MXFP8, and 3.01x for MXFP4, depending on model and concurrency.