Abstract
In this survey paper, we review the constructive algorithms for structure learning in feedforward neural networks for regression problems. The basic idea is to start with a small network, then add hidden units and weights incrementally until a satisfactory solution is found. By formulating the whole problem as a state-space search, we first describe the general issues in constructive algorithms, with special emphasis on the search strategy. A taxonomy, based on the differences in the state transition mapping, the training algorithm, and the network architecture, is then presented.
Keywords
ConstructiveFeedforward neural networkComputer scienceArtificial neural networkAlgorithmArtificial intelligenceMachine learningFeed forwardState spaceMathematics
Affiliated Institutions
Related Publications
Neural Network for Graphs: A Contextual Constructive Approach
This paper presents a new approach for learning in structured domains (SDs) using a constructive neural network for graphs (NN4G). The new model allows the extension of the inpu...
Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes
According to conventional neural network theories, single-hidden-layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal appro...
The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks
To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN's) has focused primarily on extracting rule-based explan...
Training Recurrent Networks by Evolino
In recent years, gradient-based LSTM recurrent neural networks (RNNs) solved many previously RNN-unlearnable tasks. Sometimes, however, gradient information is of little use for...
An empirical evaluation of deep architectures on problems with many factors of variation
Recently, several learning algorithms relying on models with deep architectures have been proposed. Though they have demonstrated impressive performance, to date, they have only...
Publication Info
- Year
- 1997
- Type
- article
- Volume
- 8
- Issue
- 3
- Pages
- 630-645
- Citations
- 491
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
491
OpenAlex
Cite This
Tin-Yau Kwok,
Dit-Yan Yeung
(1997).
Constructive algorithms for structure learning in feedforward neural networks for regression problems.
IEEE Transactions on Neural Networks
, 8
(3)
, 630-645.
https://doi.org/10.1109/72.572102
Identifiers
- DOI
- 10.1109/72.572102