Abstract
Various techniques of optimizing criterion functions to train neural-net classifiers are investigated. These techniques include three standard deterministic techniques (variable metric, conjugate gradient, and steepest descent), and a new stochastic technique. It is found that the stochastic technique is preferable on problems with large training sets and that the convergence rates of the variable metric and conjugate gradient techniques are similar.
Keywords
Computer scienceArtificial neural networkArtificial intelligenceTraining (meteorology)Machine learning
Affiliated Institutions
Related Publications
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
Stochastic Gradient Descent (SGD) is a popular algorithm that can achieve state-of-the-art performance on a variety of machine learning tasks. Several researchers have recently ...
Highway Networks
There is plenty of theoretical and empirical evidence that depth of neural networks is a crucial ingredient for their success. However, network training becomes more difficult w...
Training Very Deep Networks
Theoretical and empirical evidence indicates that the depth of neural networks is crucial for their success. However, training becomes more difficult as depth increases, and tra...
Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression
Bounding box regression is the crucial step in object detection. In existing methods, while ℓn-norm loss is widely adopted for bounding box regression, it is not tailored to the...
Object Detection with Discriminatively Trained Part-Based Models
We describe an object detection system based on mixtures of multiscale deformable part models. Our system is able to represent highly variable object classes and achieves state-...
Publication Info
- Year
- 1992
- Type
- article
- Volume
- 3
- Issue
- 2
- Pages
- 232-240
- Citations
- 210
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
210
OpenAlex
8
Influential
166
CrossRef
Cite This
Etienne Barnard
(1992).
Optimization for training neural nets.
IEEE Transactions on Neural Networks
, 3
(2)
, 232-240.
https://doi.org/10.1109/72.125864
Identifiers
- DOI
- 10.1109/72.125864
- PMID
- 18276424