Statistics Glossary → Variable importance (gain, cover, frequency)
Variable importance (gain, cover, frequency)concept
Measures in tree-based and other machine-learning models that summarize how much each predictor contributes to the model's predictive performance.
In tree models the most common measure is impurity reduction: the total (or average) improvement achieved by all splits on a variable. XGBoost reports this as 'gain'; 'cover' shows the (weighted) amount of observations affected by the variable's splits and 'frequency' how many times the variable is used in splits. Permutation importance instead measures how much model performance drops when a variable's values are shuffled, and can be applied to any model.
Impurity-based importance tends to favour continuous or many-level variables with many distinct values. Strongly correlated predictors share importance, and removing one can raise the importance of the other. Importance says how useful a variable is to the model; it does not show the direction of the effect or causation. Partial dependence plots and SHAP values complement this information.
An everyday example
A credit card company uses 20 variables in a fraud-detection model. By gain, 'transaction amount' and 'night-time transaction' are the two most important. But 'customer ID number' also scores high; this is not a real relationship but a variable with many distinct values artificially inflating the impurity measure, and permutation importance exposes it.
In MerQur
In Gradient Boosting, the 'Importance type (Gain / Total gain / Cover / Frequency)' is reported normalized to sum to 1; variable importance tables in Random Forest and Decision Tree; and permutation importance in Support Vector Machine and Neural Network.
Analyses where this concept appears
Random ForestSupport Vector Machine (SVM)Gradient BoostingDecision Tree (CART)Neural Network (MLP)