Abstract
We compare discriminative and generative learning as typied by logistic regression and naive Bayes. We show, contrary to a widelyheld belief that discriminative classiers are almost always to be preferred, that there can often be two distinct regimes of performance as the training set size is increased, one in which each algorithm does better. This stems from the observation|which is borne out in repeated experiments|that while discriminative learning has lower asymptotic error, a generative classier may also approach its (higher) asymptotic error much faster. 1
Keywords
Discriminative modelArtificial intelligenceNaive Bayes classifierLogistic regressionGenerative grammarClassifier (UML)Pattern recognition (psychology)Bayes error rateComputer scienceMachine learningBayes' theoremGenerative modelProbabilistic classificationRegressionMathematicsBayes classifierStatisticsSupport vector machineBayesian probability
Affiliated Institutions
Related Publications
Computer systems that learn: classification and prediction methods from statistics, neural nets, machine learning, and expert systems
Preface 1 Overview of Learning Systems 1.1 What is a Learning System? 1.2 Motivation for Building Learning Systems 1.3 Types of Practical Empirical Learning Systems 1.3.1 Common...
An empirical comparison of supervised learning algorithms
A number of supervised learning methods have been introduced in the last decade. Unfortunately, the last comprehensive empirical evaluation of supervised learning was the Statlo...
Pictorial structures revisited: People detection and articulated pose estimation
Non-rigid object detection and articulated pose estimation are two related and challenging problems in computer vision. Numerous models have been proposed over the years and oft...
HMM-based passage models for document classification and ranking
We present an application of Hidden Markov Models to supervised document classification and ranking. We consider a family of models that take into account the fact that relevant...
Integrating multiple protein-protein interaction networks to prioritize disease genes: a Bayesian regression approach
BackgroundThe identification of genes responsible for human inherited diseases is one of the most challenging tasks in human genetics. Recent studies based on phenotype similari...
Publication Info
- Year
- 2001
- Type
- article
- Volume
- 14
- Pages
- 841-848
- Citations
- 1881
- Access
- Closed
External Links
Citation Metrics
1881
OpenAlex
Cite This
Andrew Y. Ng,
Michael I. Jordan
(2001).
On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes.
, 14
, 841-848.