MerQuR+Statistics Glossary

Statistics Glossary → Robust regression estimators (MM, S, LTS) and the breakdown point

Robust regression estimators (MM, S, LTS) and the breakdown pointconcept

Estimators that limit how far outlying observations and leverage points can distort regression coefficients, and the breakdown point, which shows how much contamination an estimator can withstand.

The breakdown point is the share of the data that can be replaced by bad observations before the estimate can be driven arbitrarily far off. For least squares it is zero: a single extreme observation can pull the line anywhere. M-estimators (Huber, bisquare) iteratively down-weight observations with large residuals and resist outliers in the y direction, but they offer no protection against leverage points in the x space. LTS (least trimmed squares) minimizes only the sum of the h smallest squared residuals, and the S-estimator minimizes a robust scale of the residuals. Both have breakdown points close to 50% but low efficiency for normal data.

The MM-estimator combines the two properties: it starts from a high-breakdown S estimate and then applies a highly efficient M step, achieving a 50% breakdown point together with 95% efficiency under normal errors, which is why it is the recommended choice in most cases. Robust methods do not replace investigating why outliers occur; observations that receive low weights should be examined separately. If robust and classical estimates diverge clearly, that in itself is important information about the data.

An everyday example

An estate agent studies the relationship between house size and price using 50 sales. During data entry, the prices of two villas lost a zero: they look very large but very cheap. These two leverage points pull the least-squares slope almost to zero. The M-estimator stays exposed to them as well, whereas the MM-estimator gives the two villas almost zero weight and recovers the slope shown by the other 48 sales.

In MerQur

In Robust Regression, the 'Estimator (MM, M, S, LTS)' option (MM by default, with a 50% breakdown point and 95% efficiency) and the 'Scale estimator (MAD, Huber proposal 2)' option; the scale estimator only matters for the M-estimator.

Analyses where this concept appears

Robust Regression (M-estimator)

Related concepts

M-estimator (Huber, Tukey biweight)Minimum covariance determinant (MCD) and robust distance