Skip to main content
Search

Julien Siems

1 related articles

Subscribe to updates
  1. arXiv

    Making KDA's gate signed lets a single CKDA layer track finite groups and extrapolate periodic waveforms, with 1.3B downstream accuracy on par with KDA

    The work introduces Complex KDA (CKDA): extending Kimi Delta Attention by allowing signed gate entries and an extended delta-rule coefficient range so that a single diagonal-plus-rank-one transition can realize a 2D rotation; the authors prove every orthogonal diagonal-plus-rank-one matrix is exactly a CKDA transition, show one CKDA layer tracks every finite group isomorphic to a subgroup of O(2), save one layer relative to comparable diagonal-plus-rank-one linear RNNs, and report experiments on group-word problems, periodic audio continuation, and language modeling.

You’re all caught up