# State-Space Models

3 related articles

Subscribe to updates

arXiv Nov 4, 2026
Diffusion models move temporal structure into the noising process: DSSD lowers forecast error on weather fields and robot videos
Diffusion models for spatiotemporal prediction usually add independent noise to each future frame and leave temporal dependence to the denoiser. DSSD instead builds the temporal operator of the corruption process from a learnable diagonal state-space model whose decay rates and frequencies train jointly with the denoiser, and uses the Cholesky factor of that model's stationary covariance so the operator stays causal, exactly invertible and unit-variance per frame. On SEVIR, Turbulence, BAIR and RoboNet it beats diffusion baselines sharing the same denoiser, with raw CRPS 3.975 versus 4.174 for the strongest SEVIR baseline, DPM.
Diffusion models for spatiotemporal prediction usually add independent noise to each future frame and leave temporal dependence to the denoiser. DSSD instead builds the temporal operator of the corruption process from a learnable diagonal state-space model whose decay rates and frequencies train jointly with the denoiser, and uses the Cholesky factor of that model's stationary covariance so the operator stays causal, exactly invertible and unit-variance per frame. On SEVIR, Turbulence, BAIR and RoboNet it beats diffusion baselines sharing the same denoiser, with raw CRPS 3.975 versus 4.174 for the strongest SEVIR baseline, DPM.
Seoul National University
Woojun Lee（First author）
Dynamical State-Space Diffusion State-Space Models Spatiotemporal Prediction
arXiv Oct 1, 2026
A survey recasts attention evolution as contextual-memory organization, using 59 release records and 11 open-weight endpoints
Treating model-internal contextual memory as the shared unit of analysis, this survey introduces five dimensions—Memory Representation, Memory Update, Access, Readout, and Integration—and reviews five research lines (Softmax Attention, Sparse Attention, Linear Attention, State Space Models, and Hybrid Architectures), using 59 release-level records across 14 major model lineages and a frozen comparison of 11 high-performing open-weight endpoints through September 22, 2026 to argue that attention design is diversifying rather than converging, with hybridization and cross-layer reuse making network depth a dimension along which contextual memory is organized.
Treating model-internal contextual memory as the shared unit of analysis, this survey introduces five dimensions—Memory Representation, Memory Update, Access, Readout, and Integration—and reviews five research lines (Softmax Attention, Sparse Attention, Linear Attention, State Space Models, and Hybrid Architectures), using 59 release-level records across 14 major model lineages and a frozen comparison of 11 high-performing open-weight endpoints through September 22, 2026 to argue that attention design is diversifying rather than converging, with hybridization and cross-layer reuse making network depth a dimension along which contextual memory is organized.
Zhentao Tan（First author）
Contextual Memory Attention Mechanism Hybrid Architecture Linear Attention Sparse Attention
arXiv Oct 1, 2026
FRAC swaps exponential forgetting for power-law long memory in SSMs, beating Mamba and GDN on long-context 1.3B language modeling
The work introduces FRAC, a selective state space model architecture derived from fractional dynamics that approximates a heavy-tailed fractional kernel with a finite-state, log-spaced sum of exponential modes, replacing exponential forgetting with power-law long memory and improving long-context performance over SSM baselines such as Mamba2, GDN, and Mamba3 on synthetic long-tail and recall tasks, 1.3B-parameter language modeling, and DNA modeling, while staying competitive on short-context tasks.
The work introduces FRAC, a selective state space model architecture derived from fractional dynamics that approximates a heavy-tailed fractional kernel with a finite-state, log-spaced sum of exponential modes, replacing exponential forgetting with power-law long memory and improving long-context performance over SSM baselines such as Mamba2, GDN, and Mamba3 on synthetic long-tail and recall tasks, 1.3B-parameter language modeling, and DNA modeling, while staying competitive on short-context tasks.
Ivan Kobyzev（First author）
Frac Fractional State Space Transition State-Space Models Long-Context Modeling Language modeling

You’re all caught up

Diffusion models move temporal structure into the noising process: DSSD lowers forecast error on weather fields and robot videos

A survey recasts attention evolution as contextual-memory organization, using 59 release records and 11 open-weight endpoints

FRAC swaps exponential forgetting for power-law long memory in SSMs, beating Mamba and GDN on long-context 1.3B language modeling