Statistics Glossary → Maximum likelihood (ML) estimation
Maximum likelihood (ML) estimationconcept
An estimation method that chooses the model parameters as the values that make the observed data most probable (maximize the likelihood).
A probability model gives, for each value of the parameters, the probability of observing the data at hand; viewed as a function of the parameters, this is called the likelihood. Maximum likelihood estimation finds the parameter values that maximize this function (in practice, its logarithm). In linear regression with normal errors ML gives the same coefficients as least squares; in most other models, such as logistic regression, mixed models, factor analysis and survival models, the solution is found by iterative numerical optimization.
In large samples ML estimators are consistent, approximately normally distributed and have the smallest possible variance; standard errors come from the curvature of the likelihood surface (the information matrix). The likelihood ratio test, AIC and BIC are all built on this framework. In small samples ML can be biased: for instance it divides the variance by n and so slightly underestimates it, which is why REML is often used for mixed models. If the model is misspecified or the data show separation, the estimates may fail to converge or drift to infinity.
An everyday example
At a junction, the number of cars running a red light is counted in five separate hours: 3, 5, 4, 2 and 6. If the hourly count is assumed to follow a Poisson distribution with mean λ, the value of λ that makes these five counts most probable is 4, the sample mean. Maximum likelihood generalizes this reasoning to far more complex models.
In MerQur
It is the estimation method behind many analyses, including logistic regression, mixed models, factor and latent class analysis, structural equation models and parametric survival. It is selected directly through the 'Method' option of Generalized Linear Mixed Models (Maximum likelihood - adaptive Gauss-Hermite, or Variational Bayes), 'Estimation method' in ARIMA Forecast (ML, CLS, ULS) and in the SAR model (ML or GMM), 'Estimation method (REML, ML)' in Linear Mixed Models, the 'τ² estimator' in Meta-Analysis and 'Model' (Heckman, maximum likelihood) in Sample Selection and Truncated Regression.
Analyses where this concept appears
Meta-AnalysisLogistic RegressionLinear Mixed Models (LMM)Generalized Linear Mixed Models (GLMM)Instrumental Variables Regression (2SLS)ROC CurveTrue Skill Statistic (TSS)Latent Class Analysis (LCA)Latent Profile Analysis (LPA)Cronbach's Alpha and Reliability AnalysisExploratory Factor Analysis (EFA)Confirmatory Factor Analysis (CFA)Structural Equation Modeling (SEM)Item Response Theory (IRT / Rasch)Variance Components Analysis (VARCOMP)Probit RegressionTobit Regression (Censored Regression)SAR (Spatial Autoregressive Lag)Spatial Error Model (SEM)Path AnalysisParametric Survival (AFT)Survey-PHREG (Weighted Cox)Interval-Censored SurvivalARIMA ForecastCointegration (Johansen / Engle-Granger)Unobserved Components Model (Structural Time Series)Sample Selection and Truncated Regression (Heckman)
Related concepts
Restricted maximum likelihood (REML)Likelihood ratio test (LRT)Information criteria (AIC, AICc, BIC)