MerQuR+Statistics Glossary

Statistics Glossary → Choosing the number of clusters

Choosing the number of clustersconcept

Deciding how many groups to split the data into in clustering or latent class analysis, guided by indicators such as the elbow, silhouette, gap statistic or information criteria.

Methods such as k-means need the number of clusters (K) from the user, and in hierarchical clustering one must choose how many clusters to cut the tree into. Common indicators are the elbow method, which looks for a clear bend in the decline of the within-cluster sum of squares as K grows; the average silhouette, which measures on a scale from -1 to 1 how much closer each observation is to its own cluster than to others, with the K giving the highest value chosen; the gap statistic, which compares the observed within-cluster dispersion with that in structureless (uniform) reference data; and the Calinski-Harabasz criterion, which looks at the ratio of between- to within-cluster variance. In latent class and latent profile analysis, models with different numbers of classes are compared with information criteria such as BIC.

These indicators often disagree, and there is no guarantee that a single 'true' K exists in the data; sometimes there are no distinct clusters at all. The choice should therefore weigh the statistical indicators together with how interpretable the clusters are and what they will be used for. Cluster stability can be checked by seeing whether the same structure emerges from different starting points or subsamples. The scaling of the variables and the distance measure chosen also influence the K that is suggested.

An everyday example

A clothing brand wants to group its customers by spending and shopping frequency. With K = 2 it gets 'low spenders' and 'high spenders'; with K = 4 it also gets 'rare but big-ticket' and 'frequent but small-basket' groups. If the silhouette is highest at K = 4 and the four groups translate into sensible campaigns for the marketing team, four clusters is a reasonable choice.

In MerQur

In K-Means Clustering, 'Number of clusters (K)', the 'K selection criterion' (manual, highest mean silhouette, gap statistic) and the 'K-selection criteria table' (elbow, silhouette, Calinski-Harabasz); in Hierarchical Clustering, 'Number of clusters (K)' and the 'K-selection criteria table'; in Variable Clustering, 'Number of clusters (K)'. Latent Class and Latent Profile Analysis compare models with 1..K classes using fit criteria.

Analyses where this concept appears

Variable Clustering (VARCLUS)K-Means ClusteringHierarchical ClusteringDBSCAN / HDBSCANLatent Class Analysis (LCA)Latent Profile Analysis (LPA)Hexbin MapDBSCAN Spatial Clustering

Related concepts

Silhouette coefficientGap statisticElbow method and k-distance plot