Statistics Glossary → Subsample ratio (stochastic boosting)
Subsample ratio (stochastic boosting)concept
A setting in gradient boosting that makes each new tree be built on a random portion of the training data rather than all of it, and that sets the size of that portion.
Gradient boosting adds trees one after another, each trying to correct the errors of the previous ones. In Friedman's (2002) stochastic gradient boosting, only a share of the rows, for example 50% to 80%, is drawn at random without replacement in each round, and the tree is built on that subsample. With a ratio of 1 all the data are used and the method is deterministic.
Subsampling injects randomness between trees. This usually reduces overfitting, improves generalisation and speeds up computation. Very small ratios, however, make each tree noisy and call for more rounds or a smaller learning rate. When the ratio is below 1, the result depends on the random seed, which should therefore be reported. It is tuned together with the learning rate and the number of trees, preferably by cross-validation. It differs from the bootstrap sampling of random forests, where sampling is with replacement and the trees are built independently.
An everyday example
An insurer predicts claims from 50,000 policies. With a subsample ratio of 0.7, each new tree is built on 35,000 randomly chosen policies. Because the trees are not identical copies of each other, the model memorises fewer chance patterns in the training data and predicts new policies better.
In MerQur
In gradient boosting, the 'Subsample ratio' sets the share of rows used in each round; when it is below 1 the result depends on the seed.