Statistics Glossary → Generalized cross-validation (GCV)
Generalized cross-validation (GCV)concept
A criterion that gives a fast approximation to leave-one-out cross-validation for linear smoothers and penalized regression, used to choose the smoothing or penalty parameter.
For methods whose fitted values can be written as ŷ = H·y (ridge regression, splines, LOESS), the effective number of parameters is the trace of H (tr(H)). GCV = n · RSS / (n − tr(H))²: as the model becomes more flexible the residual sum of squares falls, but the denominator shrinks too, so overfitting is penalized. Proposed by Craven and Wahba, it approximates leave-one-out cross-validation by replacing the individual leverages with their average.
It is fast because it comes from a single fit, without refitting the model. The parameter with the smallest GCV is chosen. When observations are not independent (for example, autocorrelated series) it can pick curves that are too flexible, and in small samples it sometimes tends to overfit. It can be used alongside other criteria such as REML and AIC.
An everyday example
A city's daily air-pollution curve is smoothed. A very flexible curve follows every daily wiggle and the residuals almost vanish, but because tr(H) approaches the number of observations, GCV grows. A very stiff curve misses the seasonal pattern and the residuals grow. The intermediate flexibility that minimizes GCV is chosen.
In MerQur
In Ridge Regression, GCV in the 'α selection criterion' list selects the alpha that minimizes RSS/(n - df)², and 'Extra output: HKB/L-W/GCV' adds extra rows. In the Generalized Additive Model ('Smoothing selection (Fixed λ, Automatic GCV/UBRE, Automatic REML)') and LOESS ('Span selection (Fixed (manual), Automatic: AICc, Automatic: GCV)') it sets the amount of smoothing from the data.
Analyses where this concept appears
Ridge RegressionGeneralized Additive Model (GAM, Nonlinear Regression)LOESS (Local Regression)