Statistics Glossary → Early exaggeration (t-SNE)
Early exaggeration (t-SNE)concept
A t-SNE setting that multiplies the high-dimensional similarity probabilities by a factor during the first phase of optimization, so that clusters separate early in the low-dimensional map.
t-SNE keeps points that are close in high-dimensional space close in the two-dimensional map by minimizing the Kullback-Leibler divergence between probability distributions (P in high dimensions, Q in low dimensions). At the start of optimization the points are scattered randomly, so the clusters begin intermixed. In early exaggeration, the P probabilities are multiplied by a factor (usually 12) for the first few hundred iterations; this artificially strengthens the attraction between neighbouring points and lets tight, well-separated clusters form quickly. When this phase ends the factor returns to 1 and the map is refined in detail.
Early exaggeration affects the overall layout of the final map, especially the gaps between clusters; very small values can leave clusters entangled, and very large values can make optimization unstable. Its effect interacts with the learning rate and perplexity. Whatever value is chosen, distances between clusters and cluster sizes in a t-SNE map should not be interpreted directly; the method is designed to preserve local neighbourhoods.
An everyday example
To group pupils in a schoolyard by class, a teacher shouts loudly in the first minutes, 'everyone run to your own class!'. Thanks to this exaggerated call the groups separate quickly; then the teacher lowers her voice and calmly arranges the order within each group.
In MerQur
Available as the 'Early exaggeration' option in t-SNE; it sets the factor by which the P probabilities are multiplied in the first phase.
Analyses where this concept appears
t-SNE (t-Distributed Stochastic Neighbor Embedding)
Related concepts
Perplexity (t-SNE effective number of neighbours)Learning rate (shrinkage)