Abstract
A new method, called the nonnegative (nn) garrote, is proposed for doing subset regression. It both shrinks and zeroes coefficients. In tests on real and simulated data, it produces lower prediction error than ordinary subset selection. It is also compared to ridge regression. If the regression equations generated by a procedure do not change drastically with small changes in the data, the procedure is called stable. Subset selection is unstable, ridge is very stable, and the nn-garrote is intermediate. Simulation results illustrate the effects of instability on prediction error.
Keywords
RegressionRidgeMathematicsSelection (genetic algorithm)Regression analysisStatisticsMean squared prediction errorLinear regressionStepwise regressionStability (learning theory)Applied mathematicsComputer scienceArtificial intelligenceMachine learningGeology
Affiliated Institutions
Related Publications
Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR
Summary Variable selection can be challenging, particularly in situations with a large number of predictors with possibly high correlations, such as gene expression data. In thi...
Regression Shrinkage and Selection Via the Lasso
SUMMARY We propose a new method for estimation in linear models. The ‘lasso’ minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients b...
Collinearity: a review of methods to deal with it and a simulation study evaluating their performance
Collinearity refers to the non independence of predictor variables, usually in a regression‐type analysis. It is a common feature of any descriptive ecological data set and can ...
On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics
A new approach to problems of multiple significance testing was presented in Benjamini and Hochberg (1995), which calls for controlling the expected ratio of the number of erron...
The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation
Regression analysis makes up a large part of supervised machine learning, and consists of the prediction of a continuous independent target from a set of other predictor variabl...
Publication Info
- Year
- 1995
- Type
- article
- Volume
- 37
- Issue
- 4
- Pages
- 373-373
- Citations
- 785
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
785
OpenAlex
Cite This
Leo Breiman
(1995).
Better Subset Regression Using the Nonnegative Garrote.
Technometrics
, 37
(4)
, 373-373.
https://doi.org/10.2307/1269730
Identifiers
- DOI
- 10.2307/1269730