Statistics Glossary → F1 score, MCC and balanced accuracy
F1 score, MCC and balanced accuracyconcept
Three classification performance measures computed from the confusion matrix, used to avoid the misleading picture plain accuracy gives when classes are imbalanced.
In data where events make up 5%, a model that says 'no event' for every case achieves 95% accuracy yet is useless. The F1 score is the harmonic mean of precision (the share of positive predictions that are correct) and recall or sensitivity (the share of true positives that are caught): F1 = 2·TP / (2·TP + FP + FN). Balanced accuracy is the arithmetic mean of sensitivity and specificity and gives each class equal weight. The Matthews correlation coefficient (MCC) is the correlation between true and predicted classes; it ranges from -1 to +1 and uses all four cells of the confusion matrix.
F1 ignores true negatives entirely, so it depends on which class is called 'positive' and says nothing about performance on the negative class. Because MCC is high only when the model does well on both classes, it is often recommended as the most informative single number for imbalanced data. All three measures depend on a probability threshold; for threshold-free discrimination, AUC is used. Closed-form standard errors are not common, so confidence intervals are usually built by bootstrap.
An everyday example
An email service tests its filter on 1000 messages, 50 of which are spam. The filter catches 40 spam messages, misses 10 and wrongly flags 20 normal messages. Accuracy looks impressive at 97%, but precision is 40/60 = 0.67, recall 0.80, F1 ≈ 0.73 and MCC ≈ 0.71, which show more honestly that the filter still needs work.
In MerQur
Among the core outputs of Confusion Matrix Metrics; the 'F1/MCC/BA/Youden bootstrap CI' output gives percentile bootstrap intervals for them. They also appear in the test-set performance output of random forest, support vector machine and decision tree.
Analyses where this concept appears
Confusion Matrix MetricsRandom ForestSupport Vector Machine (SVM)Decision Tree (CART)
Related concepts
Classification table (confusion matrix)Precision-recall curve and average precision