MerQuR+Statistics Glossary

Statistics Glossary → Restricted maximum likelihood (REML)

Restricted maximum likelihood (REML)concept

A form of maximum likelihood that estimates variance components in mixed models while accounting for the degrees of freedom used up by estimating the fixed effects.

Ordinary maximum likelihood (ML) estimates variances as if the fixed effects were known and therefore systematically underestimates them; in the simplest case the ML variance divides by n, while the unbiased estimator divides by n - 1. REML builds the likelihood on combinations of the data that are free of the fixed effects (error contrasts), so the p degrees of freedom spent on the fixed effects are taken into account and the variance components are estimated with less bias. In balanced designs REML matches the classical ANOVA estimators (apart from negative values).

The REML likelihood depends on the structure of the fixed effects, so comparing two models with different fixed effects by a REML likelihood ratio test, AIC or BIC is invalid. For such comparisons the models must be fitted by ML; only models that differ in their random part can be compared under REML. The difference matters most with small samples and many fixed effects. In meta-analysis, REML is also a common estimator of the between-study variance τ².

An everyday example

The weights of parts made by five machines are measured, and the variability between machines is of interest. With only five machines, estimating the overall mean uses up a large share of the information; ML understates the between-machine variance, whereas REML accounts for this loss and gives a more realistic value.

In MerQur

'Estimation method' (REML, ML) in Linear Mixed Models and 'Estimation method' (REML (recommended), ML, ANOVA, MINQUE) in Variance Components Analysis; the nested and crossed mixed models, meta-analysis, GAM and hierarchical Bayesian regression also use REML estimation.

Analyses where this concept appears

Meta-AnalysisLinear Mixed Models (LMM)Nested Mixed Model (Nested LMM, Multi-Level Hierarchical Mixed Model)Crossed Mixed Model (Crossed LMM, Crossed Random Effects Mixed Model)Variance Components Analysis (VARCOMP)Generalized Additive Model (GAM, Nonlinear Regression)Hierarchical Bayesian Regression

Related concepts

Maximum likelihood (ML) estimationVariance components