Recurrent Networks

ICLR 2025arXiv:2402.18510

#2

RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval

Kaiyue Wen, Xingyu Dang, Kaifeng Lyu

in-context retrievalrepresentation gaprecurrent neural networkstransformer architecture+4

51

ICML 2024arXiv:2307.11888

#3

Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues

Antonio Orvieto, Soham De, Caglar Gulcehre et al.

state-space modelslinear rnnssequence modelingcomplex eigenvalues+3

35

ICLR 2024arXiv:2309.12252

#4

Parallelizing non-linear sequential models over the sequence length

Yi Heng Lim, Qi Zhu, Joshua Selfridge et al.

29

ICLR 2024arXiv:2309.08045

#5

Traveling Waves Encode The Recent Past and Enhance Sequence Learning

T. Anderson Keller, Lyle Muller, Terrence Sejnowski et al.

24

ICLR 2024arXiv:2305.19044

#6

Exploring the Promise and Limits of Real-Time Recurrent Learning

Kazuki Irie, Anand Gopalakrishnan, Jürgen Schmidhuber

20

AAAI 2025arXiv:2408.14909

#7

SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models

Shuaijie Shen, Chao Wang, Renzhuo Huang et al.

20

ICLR 2024arXiv:2402.04882

#8

LMUFormer: Low Complexity Yet Powerful Spiking Model With Legendre Memory Units

Zeyu Liu, Gourav Datta, Anni Li et al.

17

ICLR 2025arXiv:2406.02923

#9

P-SPIKESSM: HARNESSING PROBABILISTIC SPIKING STATE SPACE MODELS FOR LONG-RANGE DEPENDENCY TASKS

Malyaban Bal, Abhronil Sengupta

spiking neural networksstate space modelslong-range dependenciessequence learning tasks+4

12

ICML 2024arXiv:2403.11998

#10

Learning Useful Representations of Recurrent Neural Network Weight Matrices

Vincent Herrmann, Francesco Faccio, Jürgen Schmidhuber

recurrent neural networksweight matrix representationspermutation equivariant layersfunctionalist approach analysis+4

12

ICML 2025arXiv:2410.22391

#11

A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks

Thomas Schmied, Thomas Adler, Vihang Patil et al.

11

ICLR 2024arXiv:2302.05326

#12

Scalable Real-Time Recurrent Learning Using Columnar-Constructive Networks

Khurram Javed, Haseeb Shah, Richard Sutton et al.

10

ICML 2024arXiv:2302.05326

#13

Scalable Real-Time Recurrent Learning Using Columnar-Constructive Networks

Khurram Javed, Haseeb Shah, Richard Sutton et al.

recurrent neural networksreal-time recurrent learninggradient-based methodsonline learning updates+3

10

ICML 2025arXiv:2507.02782

#14

Understanding and Improving Length Generalization in Recurrent Models

Ricardo Buitrago Ruiz, Albert Gu

9

ICLR 2024arXiv:2309.12927

#15

Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks

Sina Khajehabdollahi, Roxana Zeraati, Emmanouil Giannakakis et al.

recurrent neural networkslong timescalesmemory taskstraining curriculum+4

9

AAAI 2025arXiv:2311.04830

#16

Real-Time Recurrent Reinforcement Learning

Julian Lemmel, Radu Grosu

partially observable mdpsmeta-reinforcement learningtemporal difference learningeligibility traces+4

6

NEURIPS 2025arXiv:2505.21717

#17

Scaling Up Liquid-Resistance Liquid-Capacitance Networks for Efficient Sequence Modeling

Mónika Farsang, Radu Grosu

sequence modelingrecurrent modelsstate-space layersgradient stability+3

5

AAAI 2025arXiv:2412.13023

#18

Relational Neurosymbolic Markov Models

Lennert De Smet, Gabriele Venturato, Luc De Raedt et al.

4

NEURIPS 2025arXiv:2507.14793

#19

Flow Equivariant Recurrent Neural Networks

Andy Keller

flow equivariancerecurrent neural networkslie subgroupssequence models+3

4

NEURIPS 2025arXiv:2506.02475

#20

Improving Bilinear RNN with Closed-loop Control

Jiaxi Hu, Yongqi Pan, Jusen Du et al.

bilinear rnnsclosed-loop controlsequence modelingrecurrent memory management+4

AAAI 2025arXiv:2404.15597

#21

GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL

Lang Qin, Ziming Wang, Runhao Jiang et al.

spiking neural networksreinforcement learningpartially observable mdpsmulti-agent reinforcement learning+4

COLM 2025arXiv:2505.07793

#22

Overflow Prevention Enhances Long-Context Recurrent LLMs

Assaf Ben-Kish, Itamar Zimerman, Muhammad Jehanzeb Mirza et al.

MambaSub-Quadratic ModelsLong ContextLong-Range Language Modeling+1

COLM 2025arXiv:2503.22913

#23

Resona: Improving Context Copying in Linear Recurrence Models with Retrieval

Xinyu Wang, Linrui Ma, Jerry Huang et al.

Large Language ModelsLinear Recurrent ModelsIn-Context LearningRetrieval

ICML 2025arXiv:2502.17433

#24

Dynamical phases of short-term memory mechanisms in RNNs

Bariscan Kurtkaya, Fatih Dinc, Mert Yuksekgonul et al.

NEURIPS 2025arXiv:2505.21749

#25

Revisiting Bi-Linear State Transitions in Recurrent Neural Networks

Reza Ebrahimi, Roland Memisevic

bilinear operationsrecurrent neural networksstate tracking taskshidden state evolution+4

1

NEURIPS 2025arXiv:2505.13567

#26

Learning Dynamics of RNNs in Closed-Loop Environments

Yoav Ger, Omri Barak

1

NEURIPS 2025arXiv:2506.11997

#27

pLSTM: parallelizable Linear Source Transition Mark networks

Korbinian Pöppel, Richard Freinschlag, Thomas Schmied et al.

1

low-rank rnnsteacher-training settingsneural engineering frameworkorthogonal matching pursuit+4

#28

Efficient Training of Minimal and Maximal Low-Rank Recurrent Neural Networks

Anushri Arora, Jonathan Pillow

NEURIPS 2025

linear recurrencesparallel algorithmcuda extensionpytorch implementation+3

#29

Linear Recurrences Accessible to Everyone

Felix Sarnthein

ICLR 2025

NEURIPS 2025arXiv:2510.25674

#30

Recurrent Graph Neural Networks and Their Connections to Bisimulation and Logic

Maximilian Pflüger, David Tena Cucala, Egor Kostylev

Any-SSR: How Recursive Least Squares Works in Continual Learning of Large Language Model

Kai Tong, Kang Pan, Xiao Zhang et al.

Mechanistic Interpretability of RNNs emulating Hidden Markov Models

Elia Torre, Michele Viscione, Lucas Pompe et al.

recurrent neural networkshidden markov modelslatent dynamicsstochastic resonance+4

quantile regressionrecurrent neural networkssparse neural networkshierarchical interaction model+3

#33

Nonparametric Quantile Regression with ReLU-Activated Recurrent Neural Networks

Hang Yu, Lyumin Wu, Wenxin Zhou et al.

NEURIPS 2025

metric automata theoryrecurrent neural networksstate space modelsxlstm architecture+4

#34

Learning dynamics in linear recurrent neural networks

Alexandra Proca, Clémentine Dominé, Murray Shanahan et al.

Metric Automata Theory: A Unifying Theory of RNNs

Adam Dankowiakowski, Alessandro Ronca

NEURIPS 2025

NEURIPS 2025arXiv:2507.04416

#36

RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling

Xiuying Wei, Anunay Yadav, Razvan Pascanu et al.

chunk-based modelingrecurrent neural networksattention mechanismlong-range dependencies+3