# Hierarchical KV Cache

The work proposes WorldAttention, an attention architecture for interactive video world models that combines a Hierarchical KV Cache (HKV) storing and retrieving page-level KV across GPU, CPU, and NVMe tiers with a Hybrid Sparse Attention (HSA) that fuses a linear global branch and a head-adaptive block-sparse branch, reporting subject consistency of 0.9472 on VBench-Long and 0.9668 on InterVBench, a 14.02x sparse-kernel speedup over FlashAttention-3, a 2.21x end-to-end speedup, and 22.0 FPS on a single NVIDIA H100.

The work proposes WorldAttention, an attention architecture for interactive video world models that combines a Hierarchical KV Cache (HKV) storing and retrieving page-level KV across GPU, CPU, and NVMe tiers with a Hybrid Sparse Attention (HSA) that fuses a linear global branch and a head-adaptive block-sparse branch, reporting subject consistency of 0.9472 on VBench-Long and 0.9668 on InterVBench, a 14.02x sparse-kernel speedup over FlashAttention-3, a 2.21x end-to-end speedup, and 22.0 FPS on a single NVIDIA H100.