Statistics Glossary → M-estimator (Huber, Tukey biweight)
M-estimator (Huber, Tukey biweight)concept
A family of estimators of location or regression that resist outliers by using a loss function that limits the influence of large residuals.
Least squares adds up the squared residuals, so a single very extreme value can pull the estimate towards itself. M-estimators replace the square with a function that grows more slowly for large residuals. The Huber function penalizes small residuals quadratically and those beyond a tuning constant (k = 1.345 in standard-error units) linearly, so extreme values lose weight without being discarded. Tukey's biweight (bisquare) goes further and gives zero weight to residuals beyond its tuning constant (c = 4.685). Hampel's function is a piecewise compromise between the two. The estimate is found by iteratively reweighted least squares.
These standard tuning constants retain about 95% of the efficiency of least squares when the data really are normal. A smaller constant gives a more resistant but less efficient estimate. M-estimators resist outliers in the y direction, but on their own they do not protect against observations that are extreme in the predictors (high-leverage points). Redescending functions such as Tukey's can have several local solutions and therefore need a good starting estimate.
An everyday example
The monthly electricity bills of five homes on a street are 410, 420, 430, 440 and 2,300 lira; the last one is actually a workshop. The arithmetic mean jumps to 800 lira and represents none of the homes. A Huber M-estimator down-weights the extreme bill and gives something around 430, while Tukey's biweight ignores it altogether and lands near 425.
In MerQur
In Robust Regression, when the M estimator is chosen under 'Estimator' (MM, M, S, LTS), the 'Norm (M-estimator: Huber-T, Tukey biweight, Hampel)' and 'Tuning constant (M)' options apply; leaving the constant at 0 uses the default values with 95% efficiency under the normal distribution (1.345 / 4.685). In Descriptive Statistics, 'Robust measures (5% trimmed mean, Huber M)' gives the Huber M location estimate, and the 'Robust' option in STL Seasonal Decomposition and the 'Symmetric (robust, bisquare)' family in LOESS use bisquare weighting.
Analyses where this concept appears
Descriptive StatisticsRobust Regression (M-estimator)STL Seasonal DecompositionLOESS (Local Regression)
Related concepts
Trimmed meanRobust regression estimators (MM, S, LTS) and the breakdown pointOutlier