Abstract
It is shown that training a neural network using a mean-square-error criterion gives network outputs that approximate posterior class probabilities. Based on this probabilistic interpretation of the network operation, information-theoretic training criteria such as maximum mutual information and the Kullback-Liebler measure are investigated. It is shown that both of these criteria are equivalent to the maximum-likelihood estimation (MLE) of the network parameters. MLE of a network allows for the comparison of network models using the Akaike information criterion and the minimum-description length criterion.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">></ETX>
Keywords
Akaike information criterionProbabilistic logicMinimum description lengthArtificial neural networkArtificial intelligenceComputer scienceClass (philosophy)Probabilistic neural networkMaximum likelihoodBayesian information criterionMean squared errorMutual informationMachine learningMeasure (data warehouse)Interpretation (philosophy)Data miningPattern recognition (psychology)MathematicsStatisticsTime delay neural network
Related Publications
Neural network ensembles
Several means for improving the performance and training of neural networks for classification are proposed. Crossvalidation is used as a tool for optimizing network parameters ...
Low-density parity-check codes
A low-density parity-check code is a code specified by a parity-check matrix with the following properties: each column contains a small fixed number <tex xmlns:mml="http://www....
Probabilistic visual learning for object detection
We present an unsupervised technique for visual learning which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. Two types of density ...
Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling
Acoustic models used in hidden Markov model/neural-network (HMM/NN) speech recognition systems are usually trained with a frame-based cross-entropy error criterion. In contrast,...
Asymptotically optimal estimation of MA and ARMA parameters of non-Gaussian processes from high-order moments
A description is given of an asymptotically-minimum-variance algorithm for estimating the MA (moving-average) and ARMA (autoregressive moving-average) parameters of non-Gaussian...
Publication Info
- Year
- 2002
- Type
- article
- Pages
- 1361-1364
- Citations
- 173
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
173
OpenAlex
Cite This
H. Gish
(2002).
A probabilistic approach to the understanding and training of neural network classifiers.
International Conference on Acoustics, Speech, and Signal Processing
, 1361-1364.
https://doi.org/10.1109/icassp.1990.115636
Identifiers
- DOI
- 10.1109/icassp.1990.115636