Statistics Glossary → Number of trees (ensemble models)
Number of trees (ensemble models)concept
The number of decision trees grown in ensemble methods such as random forests, gradient boosting and Isolation Forest; it plays a different role depending on the method.
In a random forest, trees are grown independently on bootstrap samples and their predictions are averaged (or voted). As the number of trees grows, the variance of the predictions falls and the error settles at a stable level; adding trees does not cause overfitting, it only costs computing time. For measures such as variable importance and the out-of-bag (OOB) error to be stable, more trees are often needed than for prediction alone.
In gradient boosting, by contrast, trees are grown in sequence, each correcting the errors of the previous ones; here the number of trees (rounds) directly sets the model's complexity, and too many can overfit. The right number is chosen together with the learning rate: a smaller learning rate needs more rounds. The number of rounds is therefore usually set by cross-validation or early stopping.
An everyday example
A property website predicts house prices with a random forest. With 50 trees the test error differs slightly on each run; with 500 trees it settles at around 41 000, and 2000 trees bring no improvement. In gradient boosting, however, the test error starts rising again after 300 rounds: the model has begun to memorize the training data.
In MerQur
'Number of trees' (ntree) in Random Forest; in Gradient Boosting the 'Number of trees, Learning rate (eta), Maximum depth' parameters set the number of rounds; in Anomaly Detection, 'Isolation Forest trees and seed' sets the number of trees in the forest.
Analyses where this concept appears
Random ForestGradient BoostingAnomaly Detection