IBM opens the Spyre backend compiler: Inductor's core split becomes a dataflow pipeline
Synopsis
IBM has open-sourced a dataflow scheduler for its Spyre accelerator that reads KTIR, an MLIR representation, and turns Inductor's already-settled core division, scratchpad residency and access tiles into a dataflow pipeline, which device-specific stages then lower to SpyreCode. Previously no interface carried the full data-parallel view, so the backend had to rediscover the mapping. Bringing up the first op was about 4x faster than on the previous stack and later ops about 8x; coverage is elementwise operations with broadcast and reduction dimensions.
Interpretation
Spyre's compilation path now has an explicit middle contract: the Inductor frontend decides which core computes what and where data lives, and the backend only schedules dataflow inside those decisions. Previously Inductor's layout, tiling, work division and scratchpad allocation lacked an interface that could express the accelerator's full data-parallel view, so the backend could not simply inherit them. KTIR maps each class of Inductor decision onto a construct, including a grid attribute, explicit memory spaces, and distributed memory views for tensors split across cores.
The scheduler starts from the simplest legal pipeline and refines it into one that runs on the hardware and is larger. The starting point is a three-stage load, compute and store pipeline, but no hardware path connects device memory straight to the compute engine through a FIFO. Path expansion runs a shortest-path search over the datapaths in the architecture description and rewrites three stages into five, inserting DMA load, scratchpad load, compute, scratchpad store and DMA store, each tagged with its execution unit.
The pipeline is then nested one level of the memory hierarchy at a time, double-buffered, and split across sub-core instances so transfers overlap compute. After path expansion every stage still runs once per 64-element vector, so DMA cannot move data in bulk and the scratchpad holds no data ahead of compute. An outer pipeline works a tile at a time around an inner pipeline that works a vector at a time, scratchpad buffers read and written across stages are double-buffered, and the instance count comes from the architecture description.
All hardware knowledge lives in one architecture description file, so retargeting means writing a new description rather than changing the scheduler. Hard-coding hardware details into the passes would force changes to the scheduler itself for each new device. Spyre has its own description, the scheduler's tests run against a sample device description, and the same passes run unchanged on both.
Perspective
The demonstrated scope is elementwise operations with broadcast and reduction dimensions across multi-core KTIR programs, validated end to end from KTIR through scheduling and code generation on representative transformer operations. Working paths exist from both the Inductor and Triton frontends. Matrix multiplication support is still being completed and cost-model-driven performance optimization is being added, with the goal of running LLMs end to end.
In the V1 scheduler, loop structure, pipeline granularity, buffering depth, instance split and tile size are chosen by local heuristics rather than search or a cost model; these choices are kept explicit in the IR as placeholders, modulo factors and distribute counts so a cost model or autotuner can be added later without changing pipeline structure. The op bring-up speedups compare first and later ops and give no absolute performance figures.