MerQuR+Statistics Glossary

Statistics Glossary → Classification table (confusion matrix)

Classification table (confusion matrix)concept

A table that cross-tabulates the classes predicted by a model against the true classes and shows the kinds of correct and incorrect classifications.

For a binary outcome the table has four cells: true positives, false positives, false negatives and true negatives. From these come accuracy (the share classified correctly), sensitivity (the share of actual positives caught), specificity (the share of actual negatives correctly identified), the positive and negative predictive values (PPV, NPV) and the F1 score. With a multi-class outcome the table is k × k, and the diagonal shows the correct classifications.

For models that output probabilities (such as logistic regression), the table depends on a classification threshold; as the threshold moves, sensitivity and specificity change in opposite directions. With imbalanced classes accuracy is misleading: if 95% of cases are negative, a model that calls everyone 'negative' reaches 95% accuracy while catching no positives at all. It should therefore be judged together with balanced accuracy, the sensitivity-specificity pair or the threshold-free AUC. A table computed on the data used to build the model is optimistic.

An everyday example

An email filter classifies 1,000 messages: it catches 90 of the 100 spam messages and misses 10, and it wrongly moves 30 of the 900 normal messages to the spam folder. Sensitivity is 90%, specificity about 96.7% and accuracy 96%. But only 90 of the 120 messages flagged as spam really are spam (PPV = 75%).

In MerQur

'Extra output: Classification + AUC' and 'Classification cutoff' in Logistic Regression; 'Classification table' in multinomial and ordinal logistic, probit and weighted logistic regression; 'Additional output: Multi-class matrix' in Confusion Matrix Metrics; the hit rate in the conditional logit. The True Skill Statistic gives the confusion matrix and Discriminant Analysis the classification table; in discriminant analysis, 'Leave-one-out (LOO) cross-validation' adds a cross-validated classification table.

Analyses where this concept appears

Logistic RegressionMultinomial Logistic RegressionOrdinal Logistic RegressionTrue Skill Statistic (TSS)Confusion Matrix MetricsDiscriminant Analysis (LDA / QDA)Weighted Logistic Regression (Survey Logistic Regression)Probit RegressionConditional Logit (Choice Model)

Related concepts

Classification threshold (cut-off)Sensitivity and specificityF1 score, MCC and balanced accuracy