MerQuR+Statistics Glossary

Statistics Glossary → Coefficient path and ridge trace

Coefficient path and ridge traceconcept

A plot, used in regularized regression, of how each coefficient changes as the penalty parameter is varied over a grid of values.

In lasso, ridge and elastic net models, λ (or α) sets the strength of the penalty: at λ = 0 the estimates equal the ordinary least squares solution, and as λ grows the coefficients shrink toward zero. The coefficient path draws one curve per variable, with the penalty parameter (usually on a log scale) on the horizontal axis and the standardised coefficients on the vertical axis. In the lasso, coefficients reach exactly zero one by one, so the path shows the order in which variables leave or enter the model.

In ridge regression the coefficients approach zero but never become exactly zero. The ridge trace of Hoerl and Kennard (1970) was proposed for choosing visually a range of λ where the coefficients stabilise. Nowadays λ is usually chosen by cross-validation, and the trace is used to see what the chosen value means. Coefficients that change sign or shrink very quickly along the path often point to multicollinearity. In the lasso, which of several highly correlated variables enters first can be unstable, so reading the entry order as an importance ranking is risky.

An everyday example

A marketing team predicts sales from 15 variables using the lasso. On the coefficient path, under a very strong penalty only 'advertising budget' is non-zero. As the penalty decreases, 'season' enters first and then 'price discount'. At the point chosen by cross-validation 5 variables remain, and the team can see at which penalty level each variable comes in.

In MerQur

In ridge regression, 'Extra output: Ridge trace + CV curve' gives the standardised coefficient trace over the grid together with df(α). In lasso regression, 'Coefficient path' shows the path and the order in which variables enter the model. In regularized regression, 'Coefficient path' can be selected under the 'Chart' option.

Analyses where this concept appears

Ridge RegressionLasso RegressionRegularized Regression (LASSO / Ridge / ElasticNet)

Related concepts

Regularization (penalty) and the penalty parameter λ/α