Statistics Glossary → EM (expectation-maximization) algorithm
EM (expectation-maximization) algorithmconcept
An iterative algorithm that finds maximum likelihood estimates in models with unobserved (latent or missing) variables by alternating an expectation step and a maximization step.
In latent class analysis it is unknown which class a person belongs to, and in mixture models which component an observation comes from. If this were known, estimation would be easy. The EM algorithm starts from initial values. In the expectation (E) step it uses the current parameters to compute the posterior probability that each observation belongs to each class. In the maximization (M) step it re-estimates the parameters using those probabilities as weights. The two steps are repeated until the likelihood no longer changes meaningfully. Dempster, Laird and Rubin (1977) showed that no iteration decreases the likelihood.
EM is stable and easy to implement, but it can get stuck at a local maximum. It should therefore be run from many random starts, checking whether the best likelihood is replicated across several of them. It converges slowly, especially near the maximum. It does not give standard errors directly; they are computed separately from the information matrix or by bootstrap. It is also used for maximum likelihood estimation with missing data under the MAR assumption.
An everyday example
A café suspects that some coffee drinkers are 'rushed morning' customers and others 'leisurely afternoon' customers, but no one knows who belongs to which group. EM starts with rough group shares and averages, computes each customer's probability of belonging to each group, and then updates the group characteristics using those probabilities. After a few rounds the groups and their characteristics settle down.
In MerQur
Latent class analysis and latent profile analysis (finite Gaussian mixture) are estimated with the EM algorithm; 'Maximum EM iterations' sets the iteration limit for each start, and 'Number of random starts and Random seed' sets the number of starts that guard against local maxima. In item response theory, the MML path of the 'Estimation' option runs on EM, and in the cluster-robust Cox analysis the β and frailties of the shared gamma frailty model are estimated by EM.
Analyses where this concept appears
Latent Class Analysis (LCA)Latent Profile Analysis (LPA)Item Response Theory (IRT / Rasch)Cluster-Robust Cox (Marginal and Frailty Models)
Related concepts
Maximum likelihood (ML) estimationRandom starts and local optima