Pixel-space generators stop letting detail inflate global attention: low-frequency carriers cut ImageNet-512 FID by 2.84
Synopsis
Pixel-space generators feed the full image to a Transformer as patches, so finer detail lengthens the global sequence and raises quadratic attention cost, tying detail to communication. SpecSlot expands each spatial anchor into slots, gives every slot a fixed disjoint group of DCT coefficients, and sends only the lowest-frequency slot per anchor into the global stage while all slots mix locally. At ImageNet-512, the W256/K4 configuration lowers FID by 2.84 over the FLOPs-matched JiT control with 3.9% fewer GFLOPs, and trails JiT-B/16 by 0.69 FID at 38.2% lower cost.
Interpretation
Pixel-space generators can now adjust how finely output detail is subdivided separately from the length of the global attention sequence, instead of letting one patch grid set both. Earlier pixel-space generators treated each spatial patch as one token, so shrinking patches for detail also lengthened the global sequence; multiscale, efficient-attention and spectral-filtering routes compressed or restricted communication but kept output detail tied to a spatial patch grid. At ImageNet-512, W256/K4 lowers FID from 11.98 for the FLOPs-matched JiT-CM control to 9.14 at 135.64 GFLOPs versus the control's 141.21; it trails JiT-B/16 by 0.69 FID at 38.2% lower cost and higher throughput.
Each slot receives a fixed spectral role before generation, and that role decides whether the slot may take part in cross-anchor global exchange. Earlier global-local topologies shortened the sequence with windows, a few global tokens or learned sparsity, and communication eligibility did not follow a token's fixed output role. On ImageNet-256 with W256/K4, giving cross-anchor access to slot 1, 2 or 3 instead of the lowest-radial carrier raises mean FID to 6.91, 7.21 and 6.97, against 6.59 for the lowest-radial carrier, with allocation, graph and compute held fixed.
The same slot states can also be read out directly in pixel space, adding a local correction to the spectral prediction without extra tokens or attention edges. The best single-basis variant reaches 6.99 FID, and the dual-basis hybrid lowers it by a further 0.40 to 6.59. That comparison is measured at ImageNet-256 under the same spectral output setting, and the hybrid adds no tokens and no cross-anchor attention edges.
Perspective
The demonstrated scope is class-conditional generation on ImageNet-256 and ImageNet-512 plus a transfer to text-conditioned image generation; evaluation uses 50-step Heun sampling with a per-model swept classifier-free guidance scale, and the main ImageNet results are means over five independently trained seeds.
The authors note that the larger ImageNet-512 margin is consistent with the within-anchor allocation mechanism, but these operating points do not by themselves isolate the cause of that larger margin; the trend favoring more spatial anchors at a fixed token budget also holds only within the measured range, and the authors do not claim it generalizes.