Abstract
Bootstrap samples with noise are shown to be an effective smoothness and capacity control technique for training feedforward networks and for other statistical methods such as generalized additive models. It is shown that noisy bootstrap performs best in conjunction with weight-decay regularization and ensemble averaging. The two-spiral problem, a highly non-linear, noise-free data, is used to demonstrate these findings. The combination of noisy bootstrap and ensemble averaging is also shown useful for generalized additive modelling, and is also demonstrated on the well-known Cleveland heart data.
Keywords
Regularization (linguistics)Bootstrapping (finance)Computer scienceNoise (video)Feed forwardSmoothnessArtificial intelligenceAlgorithmPattern recognition (psychology)MathematicsEconometrics
Affiliated Institutions
Related Publications
Regularization Theory and Neural Networks Architectures
We had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units, called regularization networ...
Training with Noise is Equivalent to Tikhonov Regularization
It is well known that the addition of noise to the input data of a neural network during training can, in some circumstances, lead to significant improvements in generalization ...
An analysis of noise in recurrent neural networks: convergence and generalization
Concerns the effect of noise on the performance of feedforward neural nets. We introduce and analyze various methods of injecting synaptic noise into dynamically driven recurren...
Projection-Based Approximation and a Duality with Kernel Methods
Projection pursuit regression and kernel regression are methods for estimating a smooth function of several variables from noisy data obtained at scattered sites. Methods based ...
A Practical Bayesian Framework for Backpropagation Networks
A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible (1) objective comparisons between sol...
Publication Info
- Year
- 1996
- Type
- article
- Volume
- 8
- Issue
- 3-4
- Pages
- 355-372
- Citations
- 190
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
190
OpenAlex
Cite This
Yuval Raviv,
Nathan Intrator
(1996).
Bootstrapping with Noise: An Effective Regularization Technique.
Connection Science
, 8
(3-4)
, 355-372.
https://doi.org/10.1080/095400996116811
Identifiers
- DOI
- 10.1080/095400996116811