MerQuR+Statistics Glossary

Statistics Glossary → Elastic net and the L1 ratio

Elastic net and the L1 ratioconcept

A regularization method that shrinks regression coefficients with a mixture of the L1 (lasso) and L2 (ridge) penalties, with the share of each set by the L1 ratio.

The elastic net adds a penalty of α · [ρ · Σ|b_j| + (1 - ρ)/2 · Σ b_j²] to the sum of squares; α is the overall strength of the penalty and ρ (the L1 ratio) is the lasso share. ρ = 1 is pure lasso and ρ = 0 pure ridge. The L1 part sets some coefficients exactly to zero and so selects variables; the L2 part keeps the estimates stable.

From a group of strongly correlated variables, the lasso usually picks just one more or less arbitrarily, and when there are more variables than observations it can select at most n of them. The elastic net softens both limitations: it tends to keep or drop correlated variables together (the grouping effect). α and ρ are usually chosen by cross-validation, and the variables should be standardized before penalizing. Penalized estimates are biased, and classical p values after selection are not valid.

An everyday example

A researcher wants to predict disease severity from hundreds of gene-expression measurements; genes in the same biological pathway are closely linked. Whereas the lasso might pick one gene from each pathway and drop the rest, an elastic net with, say, an L1 ratio of 0.5 keeps the related genes together and gives a more stable selection.

In MerQur

The 'L1 ratio (ElasticNet)' (1 = LASSO, 0 = pure Ridge) and 'Select L1 ratio by CV (ElasticNet)' options of Regularized Regression.

Analyses where this concept appears

Regularized Regression (LASSO / Ridge / ElasticNet)

Related concepts

Regularization (penalty) and the penalty parameter λ/α