Skip to main content
Search

# Gated DeltaNet

4 related articles

Subscribe to updates
  1. arXiv

    TPU attention kernels stop idling: FlyWheel lifts softmax attention by up to 3.49x and reaches 81% of peak BF16 throughput

    FlyWheel is a set of TPU-native attention kernels covering both softmax attention and Gated DeltaNet linear attention. It targets matrix units left idle by repeated operand loads, vector-heavy work and dependency stalls, by coordinating data layouts, recasting triangular inversion and convolution as matrix products, and explicitly interleaving independent computation. On TPU v6e, softmax attention runs up to 3.49x faster than RPA-v3 and up to 1.56x faster than Splash Attention, reaching 81% of peak BF16 throughput.

    Zishuo Bao（First author）
    FlyWheelTPU attention kernel optimizationGated DeltaNetAttention Mechanism
  2. arXiv

    SSM decoding stops writing state every step: caching recent inputs halves state traffic and matches vLLM speculation

    State-space model decoding previously read and wrote the full recurrent state at every step, making state traffic the memory-bandwidth bottleneck and leaving speculative decoding without cheap rollback. ReplaySSM represents the state as a checkpoint plus a buffer of recent inputs, writes the checkpoint only when the buffer fills, and computes outputs directly on most steps. Implemented in vLLM for Mamba-2 and Gated DeltaNet, it speeds up standard decoding over a tuned baseline at serving batch size 256, reaches about the throughput of vLLM's speculative decoding on two 120B-class models, and separate state precisions reduce output KL divergence and downstream accuracy losses.

    Princeton UniversityNvidia (United States)
    Ze-Wei Liou（First author）
    ReplaySSMSSM DecodingSpeculative DecodingMamba-2Gated DeltaNet
  3. arXiv

    Yandex team's AsyncLLM lets Qwen 3.x models watch, think and act concurrently without fine-tuning, speeding up streaming video, games and system monitoring

    The work introduces AsyncLLM, a general framework built on the Python asyncio async/await programming model that lets pretrained LLMs observe, reason and act as concurrent coroutines, sharing memory in real time through CacheBlocks (attention KV caches and Gated DeltaNet recurrent states) and cache views, and demonstrates asynchronous operation with the same family of Qwen 3.x vision-language models on streaming video understanding, ViZDoom games and DevOps-Gym system monitoring without task-specific training.

    George Yakushev（First author）
    AsyncLLMAsynchronous LLM AgentsKV CacheGated DeltaNetStreaming Video Understanding
  4. arXiv

    LatentPort hands a 4B hybrid model's recurrent memory straight to a 9B receiver, cutting continuation loss to just 0.076 nats/token above native 9B without rereading the prefix

    In one architecture-matched Qwen3.5 4B→9B hybrid (Gated DeltaNet plus full attention) sibling pair, the work installs the source model's persistent inference state after reading a prefix — attention KV plus GDN recurrent matrices and convolution history — directly into the larger receiver with no target prefix replay; holding translated KV fixed, adding the GDN persistent-state package lowers teacher-forced NLL by 0.747 nats/token (95% paired document bootstrap CI [0.6921, 0.8047]) and improves all 64 PG19 documents, direct recurrent and convolution reuse beats the tested learned GDN maps, and a rank-4 correction with 434,176 trainable parameters brings continuation loss to only 0.076 nats/token above native 9B, JS divergence 0.022, native context recovery 0.

    Simon P Villani（First author, Corresponding author）
    LatentPortCross-Model Transfer of Recurrent MemoryGated DeltaNetKV CacheQwen3.5

You’re all caught up