MerQuR+Statistics Glossary

Statistics Glossary → Minimum leaf / node size

Minimum leaf / node sizeconcept

In decision trees, the minimum number of observations a leaf must contain (or a node must contain to be split); a setting that limits how finely the tree can grow.

A decision tree grows by splitting the data again and again; without limits it can keep splitting until each leaf holds a single observation and it has memorized the training data. The minimum leaf size allows a split only if each resulting leaf keeps at least a certain number of observations. The minimum node size for splitting sets how many observations a node must contain before a split is even attempted. In gradient boosting (XGBoost), min_child_weight plays a similar role, looking at the total Hessian weight in a leaf rather than the number of observations.

Small values give more detailed, more flexible trees that are more prone to overfitting; large values give simpler, more stable trees that may miss real relationships. In a single decision tree this setting, along with pruning, is one of the main tools for controlling overfitting. In a random forest the trees are deliberately grown deep and their variance is reduced by averaging, so the default leaf size is small (often 1 for classification). The best value can be chosen by cross-validation.

An everyday example

A shop builds a decision tree to predict which customers will respond to a promotion. With a leaf size of 1, the tree produces one-person rules such as 'a 34-year-old in a blue jacket who comes in on Tuesday at 3 pm', which are useless for new customers. With a minimum leaf size of 30, every rule rests on at least 30 customers and generalizes better.

In MerQur

It appears as 'Minimum leaf size (nodesize)' in Random Forest, 'Minimum node size to split (minsplit)' and 'Minimum leaf size (minbucket)' in Decision Tree, and 'min_child_weight' in Gradient Boosting.

Analyses where this concept appears

Random ForestGradient BoostingDecision Tree (CART)

Related concepts

Maximum tree depthTree pruning and the complexity parameter (cp)