Abstract
We will introduce a generic approach for solving problems in pattern recognition based on the synthesis of accurate multiclass discriminators from large numbers of very inaccurate "weak" models through the use of discrete stochastic processes. Contrary to the standard expectation held for the many statistical and heuristic techniques normally associated with the field, a significant feature of this method of "stochastic modeling" is its resistance to so-called "overtraining." The drop in performance of any stochastic model in going from training to test data remains comparable to that of the component weak models from which it is synthesized; and since these component models are very simple, their performance drop is small, resulting in a stochastic model whose performance drop is also small despite its high level of accuracy.
Keywords
OvertrainingStochastic modellingMathematicsHeuristicComponent (thermodynamics)Machine learningArtificial intelligenceStochastic processFeature (linguistics)Mathematical optimizationAlgorithmComputer sciencePattern recognition (psychology)Statistics
Affiliated Institutions
Related Publications
A Light CNN for Deep Face Representation With Noisy Labels
The volume of convolutional neural network (CNN) models proposed for face recognition has been continuously growing larger to better fit the large amount of training data. When ...
Feature selection for multiclass discrimination via mixed-integer linear programming
We reformulate branch-and-bound feature selection employing L/sub /spl infin// or particular L/sub p/ metrics, as mixed-integer linear programming (MILP) problems, affording con...
Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel
Support vector machines (SVMs) with the gaussian (RBF) kernel have been popular for practical use. Model selection in this class of SVMs involves two hyper parameters: the penal...
Using output codes to boost multiclass learning problems
This paper describes a new technique for solving multiclass learning problems by combining Freund and Schapire's boosting algorithm with the main ideas of Dietterich an...
An Overview of Overfitting and its Solutions
Overfitting is a fundamental issue in supervised machine learning which prevents us from perfectly generalizing the models to well fit observed data on training data, as well as...
Publication Info
- Year
- 1996
- Type
- article
- Volume
- 24
- Issue
- 6
- Citations
- 135
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
135
OpenAlex
Cite This
E. M. Kleinberg
(1996).
An overtraining-resistant stochastic modeling method for pattern recognition.
The Annals of Statistics
, 24
(6)
.
https://doi.org/10.1214/aos/1032181157
Identifiers
- DOI
- 10.1214/aos/1032181157