Abstract
A new criterion is proposed for the evaluation of variable selection procedures in multiple regression. This criterion, which we call the risk inflation, is based on an adjustment to the risk. Essentially, the risk inflation is the maximum increase in risk due to selecting rather than knowing the "correct" predictors. A new variable selection procedure is obtained which, in the case of orthogonal predictors, substantially improves on AIC, $C_p$ and BIC and is close to optimal. In contrast to AIC, $C_p$ and BIC which use dimensionality penalties of 2, 2 and $\\log n$, respectively, this new procedure uses a penalty $2 \\log p$, where $p$ is the number of available predictors. For the case of nonorthogonal predictors, bounds for the optimal penalty are obtained.
Keywords
MathematicsStatisticsCurse of dimensionalityRegressionFeature selectionInflation (cosmology)Regression analysisEconometricsModel selectionContrast (vision)Computer science
Related Publications
Adaptive Model Selection
AbstractMost model selection procedures use a fixed penalty penalizing an increase in the size of a model. These nonadaptive selection procedures perform well only in one type o...
The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction
We show that data augmentation provides a rather general formulation for the study of biased prediction techniques using multiple linear regression. Variable selection is a limi...
Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR
Summary Variable selection can be challenging, particularly in situations with a large number of predictors with possibly high correlations, such as gene expression data. In thi...
A Bayesian Information Criterion for Singular Models
Summary We consider approximate Bayesian model choice for model selection problems that involve models whose Fisher information matrices may fail to be invertible along other co...
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Variable selection is fundamental to high-dimensional statistical modeling, including nonparametric regression. Many approaches in use are stepwise selection procedures, which c...
Publication Info
- Year
- 1994
- Type
- article
- Volume
- 22
- Issue
- 4
- Citations
- 529
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
529
OpenAlex
Cite This
Dean P. Foster,
Edward I. George
(1994).
The Risk Inflation Criterion for Multiple Regression.
The Annals of Statistics
, 22
(4)
.
https://doi.org/10.1214/aos/1176325766
Identifiers
- DOI
- 10.1214/aos/1176325766