Abstract
The authors describe how a two-layer neural network can approximate any nonlinear function by forming a union of piecewise linear segments. A method is given for picking initial weights for the network to decrease training time. The authors have used the method to initialize adaptive weights over a large number of different training problems and have achieved major improvements in learning speed in every case. The improvement is best when a large number of hidden units is used with a complicated desired response. The authors have used the method to train the truck-backer-upper and were able to decrease the training time from about two days to four hours
Keywords
Artificial neural networkPiecewise linear functionComputer scienceLayer (electronics)Artificial intelligenceNonlinear systemFunction (biology)Activation functionPiecewiseTraining (meteorology)AlgorithmControl theory (sociology)Mathematics
Affiliated Institutions
Related Publications
Training Recurrent Networks by Evolino
In recent years, gradient-based LSTM recurrent neural networks (RNNs) solved many previously RNN-unlearnable tasks. Sometimes, however, gradient information is of little use for...
Exploring Strategies for Training Deep Neural Networks
Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varying functions. However, until recently...
Reducing the Dimensionality of Data with Neural Networks
High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors....
Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes
According to conventional neural network theories, single-hidden-layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal appro...
Network In Network
Abstract: We propose a novel deep network structure called In Network (NIN) to enhance model discriminability for local patches within the receptive field. The conventional con...
Publication Info
- Year
- 1990
- Type
- article
- Pages
- 21-26 vol.3
- Citations
- 1376
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
1376
OpenAlex
52
Influential
731
CrossRef
Cite This
Dat Nguyen,
Bernard Widrow
(1990).
Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights.
1990 IJCNN International Joint Conference on Neural Networks
, 21-26 vol.3.
https://doi.org/10.1109/ijcnn.1990.137819
Identifiers
- DOI
- 10.1109/ijcnn.1990.137819