Abstract

Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.

Keywords

Recurrent neural networkComputer scienceArtificial intelligenceDeep learningLong short term memoryFocus (optics)Artificial neural networkMachine learning

Affiliated Institutions

Related Publications

Long Short-Term Memory

Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We brief...

1997 Neural Computation 90535 citations

Network In Network

Abstract: We propose a novel deep network structure called In Network (NIN) to enhance model discriminability for local patches within the receptive field. The conventional con...

2014 arXiv (Cornell University) 1037 citations

Publication Info

Year
2019
Type
review
Volume
31
Issue
7
Pages
1235-1270
Citations
4793
Access
Closed

External Links

Social Impact

Social media, news, blog, policy document mentions

Citation Metrics

4793
OpenAlex

Cite This

Yong Yu, Xiaosheng Si, Changhua Hu et al. (2019). A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures. Neural Computation , 31 (7) , 1235-1270. https://doi.org/10.1162/neco_a_01199

Identifiers

DOI
10.1162/neco_a_01199