# Block-Sparse Attention

UltraTex is an end-to-end multi-view diffusion framework that scales image-guided 3D texturing from 512/768 to 2048 resolution through Background Token Dropping, Block-Sparse Attention, and Foreground-Aware VAE Decoding, and it builds G-buffer TexVerse covering over 268,000 3D assets, achieving 20.6-91.1x training speedup and 22.3-74.6x end-to-end inference speedup within the dataset's common foreground-ratio range of 10%-30%.

UltraTex is an end-to-end multi-view diffusion framework that scales image-guided 3D texturing from 512/768 to 2048 resolution through Background Token Dropping, Block-Sparse Attention, and Foreground-Aware VAE Decoding, and it builds G-buffer TexVerse covering over 268,000 3D assets, achieving 20.6-91.1x training speedup and 22.3-74.6x end-to-end inference speedup within the dataset's common foreground-ratio range of 10%-30%.

Researchers from Shanghai Jiao Tong University and ByteDance Seed propose PISA, a block-sparse attention mechanism that uses coarse-to-fine pyramid Top-K selection with LogSumExp scoring and hardware-aware Triton kernels to reduce block-selection complexity from O(N²/C) to O(N log N); across 418M, 1.47B, and 2.67B scales it matches BSA, NSA, and HiLS on language modeling and commonsense reasoning, achieves the highest average accuracy among sparse methods on six containment tasks, and speeds up block selection over BSA by 2.86x, 5.31x, and 9.95x at 64K, 128K, and 256K.

Researchers from Shanghai Jiao Tong University and ByteDance Seed propose PISA, a block-sparse attention mechanism that uses coarse-to-fine pyramid Top-K selection with LogSumExp scoring and hardware-aware Triton kernels to reduce block-selection complexity from O(N²/C) to O(N log N); across 418M, 1.47B, and 2.67B scales it matches BSA, NSA, and HiLS on language modeling and commonsense reasoning, achieves the highest average accuracy among sparse methods on six containment tasks, and speeds up block selection over BSA by 2.86x, 5.31x, and 9.95x at 64K, 128K, and 256K.