Statistics Glossary → One-standard-error (1-SE) rule
One-standard-error (1-SE) ruleconcept
In cross-validation, the rule of choosing the simplest model among those whose error lies within one standard error of the error of the best model.
In methods such as the lasso, ridge or decision trees, a tuning parameter (λ or cp) is chosen by cross-validation. For each candidate value the mean CV error and the standard error of that mean are computed. The value with the smallest error (λ_min) could be chosen; but because the CV error itself varies from sample to sample, simpler models with nearly the same error may be statistically indistinguishable from it. The 1-SE rule proposed by Breiman and colleagues (1984) picks the simplest model (largest λ, smallest tree) whose error stays below 'minimum error + the standard error at that point'.
The rule yields models with fewer variables that are more stable and easier to interpret, at very little cost in predictive accuracy. It is a heuristic for avoiding overfitting rather than a strict optimality principle. If predictive accuracy is what matters most, λ_min is preferred; if parsimony and interpretability matter, λ_1SE is. Because the random split into CV folds can affect the outcome, the seed should be fixed or the CV repeated.
An everyday example
You are buying a laptop. In benchmarks the fastest model scores 100 and is very expensive; a cheaper model scores 98, but the benchmark itself has a margin of error of ±3 points. Since the difference falls within the measurement uncertainty, choosing the cheaper, simpler laptop makes sense. The 1-SE rule chooses between models with the same logic.
In MerQur
It is offered as the 'λ selection rule (λ_min, λ_1SE)' in Lasso Regression, as the 'α/λ selection rule (Minimum CV error, 1-SE rule)' in Regularized Regression, as k-fold λ_1SE within the 'α selection criterion' in Ridge Regression and as 'Pruning (Given cp / Minimum CV error / 1-SE rule)' in the Decision Tree (CART).
Analyses where this concept appears
Ridge RegressionLasso RegressionDecision Tree (CART)Regularized Regression (LASSO / Ridge / ElasticNet)
Related concepts
k-fold cross-validationRegularization (penalty) and the penalty parameter λ/α