Kernel Logistic Regression and the Import Vector Machine

Abstract

The support vector machine (SVM) is known for its good performance in two-class classification, but its extension to multiclass classification is still an ongoing research issue. In this article, we propose a new approach for classification, called the import vector machine (IVM), which is built on kernel logistic regression (KLR). We show that the IVM not only performs as well as the SVM in two-class classification, but also can naturally be generalized to the multiclass case. Furthermore, the IVM provides an estimate of the underlying probability. Similar to the support points of the SVM, the IVM model uses only a fraction of the training data to index kernel basis functions, typically a much smaller fraction than the SVM. This gives the IVM a potential computational advantage over the SVM.

Keywords

Support vector machineStructured support vector machineKernel (algebra)Artificial intelligenceComputer scienceBinary classificationLogistic regressionMachine learningRelevance vector machinePattern recognition (psychology)Kernel methodPolynomial kernelRadial basis function kernelData miningMathematics

Affiliated Institutions

Stanford University US

Related Publications

Image Superresolution Using Support Vector Regression

Karl Ni , Truong Q. Nguyen

A thorough investigation of the application of support vector regression (SVR) to the superresolution problem is conducted through various frameworks. Prior to the study, the SV...

2007 IEEE Transactions on Image Processing 276 citations

Additive logistic regression: a statistical view of boosting (With discussion and a rejoinder by the authors)

Jerome H. Friedman , Trevor Hastie , Robert Tibshirani

Boosting is one of the most important recent developments in\nclassification methodology. Boosting works by sequentially applying a\nclassification algorithm to reweighted versi...

2000 The Annals of Statistics 6819 citations

Object Detection with Discriminatively Trained Part-Based Models

Pedro F. Felzenszwalb , Ross Girshick , David McAllester +1 more

We describe an object detection system based on mixtures of multiscale deformable part models. Our system is able to represent highly variable object classes and achieves state-...

2009 IEEE Transactions on Pattern Analysis... 9911 citations

A general framework for object detection

C. Papageorgiou , Michael B. Oren , Tomaso Poggio

This paper presents a general trainable framework for object detection in static images of cluttered scenes. The detection technique we develop is based on a wavelet representat...

2002 1428 citations

Kernel k-means

Inderjit S. Dhillon , Yuqiang Guan , Brian Kulis

Kernel k-means and spectral clustering have both been used to identify clusters that are non-linearly separable in input space. Despite significant research, these methods have ...

2004 1184 citations

Publication Info

Year: 2001
Type: article
Volume: 14
Pages: 1081-1088
Citations: 136
Access: Closed

External Links

Citation Metrics

136

OpenAlex

Cite This

APA Style

                            
                                    Ji Zhu, 
                                
                                    Trevor Hastie
                                
                            (2001). 
                            Kernel Logistic Regression and the Import Vector Machine. 
                            
                            , 14
                            
                            , 1081-1088.