Statistics Glossary → Influence diagnostics (Cook's distance, leverage, DFBETAS)
Influence diagnostics (Cook's distance, leverage, DFBETAS)concept
Diagnostic statistics that measure how much individual observations change the estimates of a regression model.
An observation can affect the model for two reasons: it may lie far from the others in the predictors (high leverage), or its response may differ greatly from what the model predicts (a large residual). Leverage (h) measures how extreme an observation is in predictor space; its average is p / n (p being the number of parameters), and values two or three times that draw attention. Cook's distance combines both, summarizing how much all fitted values change when the observation is removed. DFBETAS shows by how many standard errors each coefficient shifts when the observation is dropped, and DFFITS how much its own fitted value changes.
Rough cut-offs are 4 / n or 1 for Cook's D and 2 / √n for |DFBETAS|, but these are tools for picking observations to inspect, not firm rules. An influential observation is not necessarily an error: first check the data entry, then consider whether the observation belongs to a different sub-population. Observations should not be dropped simply because they change the result; results can be reported with and without them, or robust methods used. Several influential observations that mask one another can be missed by one-at-a-time diagnostics.
An everyday example
House prices in a neighbourhood are modelled against floor area. Most houses are 80-150 m², but one is a 600 m² mansion whose price is much lower than expected. Being so extreme in the predictor, this single house has high leverage; if removing it changes the price-per-square-metre coefficient markedly, Cook's distance and DFBETAS will flag it clearly.
In MerQur
It is offered in Multiple Linear Regression ('Extra output: Influence diagnostics': Cook's D, leverage, Mahalanobis, DFFITS, DFBETAS and a Bonferroni outlier test) and in Logistic Regression ('Extra outputs (2): Influence diagnostics': leverage, Pearson and deviance residuals, Cook's D, DFBETA). The 'Extra output: Residual diagnostics + rootogram' option of Poisson / Negative Binomial Regression gives leverage and Cook's D, and the residual outputs of Cox Regression, Survey-PHREG and Cluster-Robust Cox give dfbeta values. In Meta-Analysis, 'Sensitivity: Influence diagnostics' gives similar measures at the study level, and Robust Regression provides MCD-based leverage/outlier diagnostics.
Analyses where this concept appears
Meta-AnalysisMultiple Linear RegressionLogistic RegressionPoisson / Negative Binomial Regression (Count Regression)Robust Regression (M-estimator)Cox Regression (Proportional Hazards Model)Survey-PHREG (Weighted Cox)Cluster-Robust Cox (Marginal and Frailty Models)
Related concepts
Residual plots and residual diagnosticsOutlierMahalanobis distance