MerQuR+Statistics Glossary

Statistics Glossary → Coefficient of determination (R²) and adjusted R²

Coefficient of determination (R²) and adjusted R²concept

The proportion of the total variability in the dependent variable that is explained by the model; adjusted R² penalizes that proportion for the number of variables in the model.

R² = 1 - SS_residual / SS_total and, in linear regression, lies between 0 and 1. R² = 0.40 means that 40% of the variance in the dependent variable is explained by the predictors in the model. In simple linear regression with one predictor, R² equals the square of the Pearson correlation coefficient. When a new variable is added to the model, however, R² never decreases, even if that variable is pure noise, so it can be misleading when comparing models with different numbers of variables.

Adjusted R² = 1 - (1 - R²)(n - 1)/(n - p - 1) applies a penalty for the number of predictors p; it falls, and can even become negative, if a new variable does not contribute enough. A high R² does not show that the model is correctly specified or that the relationship is causal, and a low R² is compatible with meaningful and important effects (for example in inherently variable phenomena such as human behaviour). Because a true R² cannot be defined for logistic and other generalized models, pseudo-R² measures such as McFadden, Cox-Snell or Nagelkerke are used; these should not be read like the R² of linear regression.

An everyday example

A model explaining students' exam marks by hours of study gives R² = 0.50: half of the variation in marks is explained by study time, and the other half comes from things like sleep, prior knowledge or luck. If students' shoe size is added to the model, R² might creep up to 0.51; adjusted R², however, goes down, showing that the new variable adds nothing real.

In MerQur

It is reported as R² in the output of Multiple Linear, Ridge, Lasso, Panel Data, Instrumental Variables and Bayesian Linear Regression, Two-Way ANOVA, Path Analysis, the Support Vector Machine (SVR), Variable and Hierarchical Clustering and Meta-Analysis (meta-regression); as pseudo-R² measures in Logistic and Multinomial Logistic Regression, as Nakagawa R² in Linear Mixed Models and GLMM, and as the pooled R² in Multiple Imputation. The Generalized Additive Model provides it through the 'Adjusted R² and test against the linear model' extra output.

Analyses where this concept appears

Two-Way ANOVAVariable Clustering (VARCLUS)Meta-AnalysisMultiple Linear RegressionLogistic RegressionMultinomial Logistic RegressionRidge RegressionLasso RegressionLinear Mixed Models (LMM)Generalized Linear Mixed Models (GLMM)Panel Data RegressionInstrumental Variables Regression (2SLS)Support Vector Machine (SVM)Hierarchical ClusteringGeneralized Additive Model (GAM, Nonlinear Regression)Bayesian Linear RegressionPath AnalysisMultiple Imputation

Related concepts

Pseudo-R² (McFadden, Cox-Snell, Nagelkerke)Analysis of variance (ANOVA) table