MerQuR+Statistics Glossary

Statistics Glossary → Isolation forest and contamination rate

Isolation forest and contamination rateconcept

An anomaly detection method that finds outlying observations by how easily random splits can separate them from the rest, and the rate that sets how many observations are labelled anomalies.

The method, proposed by Liu, Ting and Zhou (2008), builds many random trees. In each tree a random variable and a random split point within its range are chosen, and the data are split until every observation is on its own. Because outliers are few and far from the rest, they are isolated after only a few splits, giving them a short path length; normal observations lie in dense regions and need many splits. The average path length is turned into an anomaly score, and the closer the score is to 1, the more unusual the observation.

The method makes no distributional assumptions and is fast for high-dimensional and large data. The scores give only a ranking, however; which observations count as 'anomalies' is set by the contamination rate c. The (1 − c) percentile of the scores becomes the threshold, and about c·n observations are flagged. The true rate is unknown and the result depends directly on its choice, so it should rest on domain knowledge and its sensitivity should be checked. It works without scaling the variables, but because it splits along the axes it may not fully capture structure in oblique directions. The result depends on the seed.

An everyday example

A bank screens 10,000 card transactions by amount, time and location. A very large transaction made at midnight in another country is isolated at the second or third split in most random trees and gets a high score. With a contamination rate of 0.01, the roughly 100 highest-scoring transactions are flagged for review.

In MerQur

In anomaly detection, 'Contamination (Isolation Forest and LOF)' sets the threshold as the (1 − c) percentile of the scores, and 'Isolation Forest trees and seed' set the number of trees in the forest and the random seed.

Analyses where this concept appears

Anomaly Detection

Related concepts

Local outlier factor (LOF)Outlier