Statistics Glossary → Classification threshold (cut-off)
Classification threshold (cut-off)concept
The boundary that decides from which value a model's predicted probability or score is treated as the 'positive' class.
Logistic regression, a random forest or a diagnostic test produces a continuous score for each observation. Turning that score into a 'yes/no' decision needs a threshold: positive if score ≥ threshold. The default of 0.5 is not always the best choice. Lowering the threshold raises sensitivity but lowers specificity, and raising it does the opposite. All classification measures, such as accuracy, precision, F1, kappa and TSS, depend on the chosen threshold.
The threshold can be chosen by criteria such as maximising Youden's index (sensitivity + specificity − 1), making sensitivity equal to specificity, matching the predicted prevalence to the observed one, or guaranteeing a given sensitivity. The soundest approach takes into account the relative costs of false positives and false negatives and the prevalence in the target population. If the threshold is chosen and performance is reported on the same data, the result is optimistic. AUC, by contrast, is a threshold-free measure of discrimination.
An everyday example
A bank uses a fraud-detection model. At a threshold of 0.5 it catches 60% of frauds and wrongly blocks 5% of legitimate transactions. At 0.2 it catches 90% of frauds but blocks 20% of legitimate transactions. Which threshold to choose depends on what a missed fraud and an unnecessarily inconvenienced customer cost the bank.
In MerQur
The true skill statistic offers a 'Threshold criterion' (Youden, Sensitivity = specificity, Predicted = observed prevalence, Max Kappa, Sensitivity ≥ 0.90) and a 'Manual threshold'; the random forest has a 'Probability threshold (binary)', the SVM a 'Probability threshold' applied to the Platt probability, and probit regression a 'Classification cut-off'. In the ROC curve, the 'Cut-off criterion' chooses the threshold by Youden's J, the point closest to (0,1), weighted Youden or a target sensitivity or specificity. The 'Classification cutoff' in logistic regression and the 'Probability threshold' in gradient boosting, the decision tree and the neural network change only the class predictions in binary classification.
Analyses where this concept appears
Logistic RegressionROC CurveTrue Skill Statistic (TSS)Random ForestSupport Vector Machine (SVM)Gradient BoostingDecision Tree (CART)Neural Network (MLP)Probit Regression
Related concepts
Youden's J index (optimal cut-off)Classification table (confusion matrix)