Arcing classifier (with discussion and a rejoinder by the author)

Abstract

Recent work has shown that combining multiple versions of unstable\nclassifiers such as trees or neural nets results in reduced test set error. One\nof the more effective is bagging. Here, modified training sets are formed by\nresampling from the original training set, classifiers constructed using these\ntraining sets and then combined by voting. Freund and Schapire propose an\nalgorithm the basis of which is to adaptively resample and combine (hence the\nacronym “arcing”) so that the weights in the resampling are\nincreased for those cases most often misclassified and the combining is done by\nweighted voting. Arcing is more successful than bagging in test set error\nreduction. We explore two arcing algorithms, compare them to each other and to\nbagging, and try to understand how arcing works. We introduce the definitions\nof bias and variance for a classifier as components of the test set error.\nUnstable classifiers can have low bias on a large range of data sets. Their\nproblem is high variance. Combining multiple versions either through bagging or\narcing reduces variance significantly.

Keywords

Affiliated Institutions

• University of California, Berkeley US

Related Publications