Skip to main content
Search

# Causal Resonant Field Mixing

1 related articles

Subscribe to updates
  1. arXiv

    ResonatorLM: Replacing Attention with Causal Resonant Field Mixing for Efficient Long-Context Language Modeling

    This work introduces ResonatorLM, which replaces self-attention with a causal resonant field mixer built from damped resonators, treating a token sequence as a driven latent field, running training and prefill through O(T log T) causal FFT convolution and decoding through a fixed-size recurrent state; in a matched setting near 6M parameters, WikiText-2 character-level test perplexity drops from 4.617 to 3.764 and accuracy rises from 55.32% to 61.31%, decode reaches a 6.47x speedup over an optimized attention block at 32K tokens, and a separate kernel-level benchmark reports 440.29x at 8K and 575.86x at 32K.

You’re all caught up