Statistics Glossary → Stepwise variable selection
Stepwise variable selectionconcept
An automatic variable-selection method that adds candidate variables to a model, or removes them, one step at a time according to a statistical criterion.
Forward selection starts from an empty model and at each step adds the variable that best meets the entry criterion (e.g. p < PIN). Backward elimination starts with all variables and at each step drops the weakest variable that meets the removal criterion (e.g. p > POUT). The stepwise method combines the two: after each addition, the variables already in the model are re-examined and any that have become weak are removed. Entry and removal decisions rely on partial F, score, likelihood-ratio or Wald tests; the removal threshold must be larger than the entry threshold to avoid cycling.
Stepwise selection is an exploratory tool with serious problems: the coefficients of the selected model are biased away from zero, standard errors and p values are overly optimistic because they ignore the selection process, R² is inflated, and small changes in the data can lead to a completely different model. In confirmatory studies variables should therefore be chosen on theoretical grounds, and in predictive studies regularization methods such as the lasso, or cross-validation, should be preferred. In hierarchical models (e.g. log-linear models), lower-order terms that make up an interaction are not removed while the interaction stays in the model.
An everyday example
A house-price model has 30 candidate variables: floor area, number of rooms, floor, aspect, distance to school and so on. Stepwise selection first adds the strongest variable (floor area), then the one that adds most among the rest, and so on, removing any variable that no longer contributes. Yet running the same procedure on data from a different year may select different variables, which shows how fragile the result can be.
In MerQur
It is offered as the 'Variable selection method (Enter, Stepwise, Forward, Backward)' with PIN and POUT in Multiple Linear Regression; as forward and backward stepwise selection based on LR, score or Wald tests in Logistic and Multinomial Logistic Regression; as 'Variable selection (Enter / Forward LR / Backward LR / AIC both directions)' in Cox regression; as 'Stepwise variable selection' with Wilks' Λ in Discriminant Analysis; and as 'Backward elimination (hierarchical)' with an 'Elimination criterion (p)' in the Log-Linear Model.
Analyses where this concept appears
Log-Linear ModelMultiple Linear RegressionLogistic RegressionMultinomial Logistic RegressionDiscriminant Analysis (LDA / QDA)Cox Regression (Proportional Hazards Model)
Related concepts
Hierarchical (blockwise) regressionInformation criteria (AIC, AICc, BIC)