Statistics Glossary → Multicollinearity (VIF, tolerance, condition index)
Multicollinearity (VIF, tolerance, condition index)concept
Strong linear relationships among the explanatory variables in a regression, which make their coefficients unstable and imprecisely estimated.
When two or more predictors carry nearly the same information, the model cannot separate their effects. The standard errors of the coefficients grow, their signs can be unexpected, and a small change in the data can change the coefficients a lot. The overall predictive power and R² of the model are usually unaffected, however; the problem lies in interpretation and in testing individual coefficients.
The variance inflation factor is VIF_j = 1 / (1 - R²_j), where R²_j is the share of the j-th predictor explained by the other predictors; tolerance is its reciprocal. VIF above 5 or 10 is a common rough threshold. Condition indices (Belsley, Kuh and Welsch) are computed from the eigenvalues of the predictors' scaled cross-product matrix; indices above 30, together with variables that have high variance proportions on the same dimension, point to harmful collinearity. Remedies include combining variables or dropping one, centring (for interaction and polynomial terms) and regularization methods such as ridge.
An everyday example
A house-price model contains both 'floor area' and 'number of rooms'; since larger houses also have more rooms, the two variables are closely linked. The model cannot tell how much of the price increase comes from floor area and how much from the number of rooms, and the coefficient for rooms may even come out negative.
In MerQur
'Extra output (2): Collinearity diagnostics' in Multiple Linear Regression (tolerance, eigenvalues, condition indices, variance proportions); 'Extra output: VIF' in Logistic Regression; 'Extra output: VIF / condition number' in Ridge Regression (OLS and ridge VIFs). 'Additional output: Assumption diagnostics' in Canonical Correlation gives the within-set VIF, 'OLS assumption diagnostics' in SAR give the condition number and VIF, and Exploratory Factor Analysis gives the determinant of the correlation matrix (< 0.00001 indicates multicollinearity).
Analyses where this concept appears
Canonical CorrelationMultiple Linear RegressionLogistic RegressionRidge RegressionExploratory Factor Analysis (EFA)SAR (Spatial Autoregressive Lag)
Related concepts
Regularization (penalty) and the penalty parameter λ/αStandardized regression coefficient (β)