MerQuR+Statistics Glossary

Statistics Glossary → Convergence and maximum iterations

Convergence and maximum iterationsconcept

Iterative algorithms stop when the change between successive steps falls below a tolerance (convergence), and an upper limit on iterations keeps them from running forever.

Many methods approach the solution step by step: maximum likelihood (Newton-Raphson, Fisher scoring), the EM algorithm, iteratively reweighted least squares (IRLS), k-means and neural-network training. At each step the change in the parameters or in the objective (log-likelihood, loss) is monitored; when it drops below the tolerance, the algorithm is said to have converged. The maximum number of iterations is the point at which the algorithm is stopped if convergence does not happen.

If the limit is reached without convergence, the results are not reliable and the software usually warns. Possible causes include a model too complex for the data, variables on very different scales, complete separation (in logistic regression), sparse cells or poor starting values. Raising the iteration limit is sometimes enough, but convergence does not guarantee that the solution found is the best one: methods such as EM and k-means can stop at a local optimum, which is why several random starts are used.

An everyday example

Someone searching for a lost key in a dark room moves each time in the direction where they sense the key is closer. When their steps shrink to a few centimetres, they stop searching (convergence). But they also set a rule of at most 100 steps; if they have not found it by then, they stop and think about starting somewhere else, because they may be circling the wrong corner.

In MerQur

'Maximum iterations' in Neural Network and K-Means Clustering; 'Maximum EM iterations' in Latent Class and Latent Profile Analysis; 'Iterations' in t-SNE; 'Max iterations (M)' in Robust Regression; 'Max iterations' and the 'Convergence trace' chart in Multiple Imputation; and 'Maximum iterations and Stop criterion' in PLS-SEM. In Nonlinear Regression, a model that fails to converge is shown under a NOT CONVERGED heading; in Lasso Regression a warning is written.

Analyses where this concept appears

Descriptive StatisticsVariable Clustering (VARCLUS)Lasso RegressionNeural Network (MLP)K-Means ClusteringLatent Class Analysis (LCA)Latent Profile Analysis (LPA)t-SNE (t-Distributed Stochastic Neighbor Embedding)Exploratory Factor Analysis (EFA)Nonlinear RegressionRobust Regression (M-estimator)Multiple ImputationLOESS (Local Regression)PLS-SEM (Partial Least Squares Path Modeling)

Related concepts

Random starts and local optimaEM (expectation-maximization) algorithm