Abstract
Abstract A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeroes coefficients. In tests on real and simulated data, it produces lower prediction error than ordinary subset selection. It is also compared to ridge regression. If the regression equations generated by a procedure do not change drastically with small changes in the data, the procedure is called stable. Subset selection is unstable, ridge is very stable, and the nn-garrote is intermediate. Simulation results illustrate the effects of instability on prediction error. KEY WORDS: Little bootstrapModel errorPredictionStability
Keywords
RegressionRidgeMathematicsSelection (genetic algorithm)Regression analysisStatisticsLinear regressionMean squared prediction errorStability (learning theory)Stepwise regressionInstabilityApplied mathematicsComputer scienceArtificial intelligenceMachine learningGeology
Affiliated Institutions
Related Publications
Better Subset Regression Using the Nonnegative Garrote
A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeroes coefficients. In tests on real and simulated data, it prod...
The Little Bootstrap and other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error
Abstract When a regression problem contains many predictor variables, it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor var...
Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR
Summary Variable selection can be challenging, particularly in situations with a large number of predictors with possibly high correlations, such as gene expression data. In thi...
A bootstrap resampling procedure for model building: Application to the cox regression model
Abstract A common problem in the statistical analysis of clinical studies is the selection of those variables in the framework of a regression model which might influence the ou...
THE LASSO METHOD FOR VARIABLE SELECTION IN THE COX MODEL
I propose a new method for variable selection and shrinkage in Cox's proportional hazards model. My proposal minimizes the log partial likelihood subject to the sum of the absol...
Publication Info
- Year
- 1995
- Type
- article
- Volume
- 37
- Issue
- 4
- Pages
- 373-384
- Citations
- 617
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
617
OpenAlex
Cite This
Leo Breiman
(1995).
Better Subset Regression Using the Nonnegative Garrote.
Technometrics
, 37
(4)
, 373-384.
https://doi.org/10.1080/00401706.1995.10484371
Identifiers
- DOI
- 10.1080/00401706.1995.10484371