MerQuR+Statistics Glossary

Statistics Glossary → Area under the ROC curve (AUC)

Area under the ROC curve (AUC)concept

The probability that a classifier or diagnostic test gives a randomly chosen positive case a higher score than a randomly chosen negative case; a summary of discriminating power across all thresholds.

The ROC curve plots sensitivity (the true positive rate) against 1 - specificity (the false positive rate) for every possible threshold. The area under it is interpreted between 0.5 and 1: 0.5 is coin-toss level and 1 is perfect discrimination; a value below 0.5 means the score works in the opposite direction. The AUC equals the Mann-Whitney U statistic divided by n₁ · n₀, which is what allows it to be read as 'the probability that a positive scores higher than a negative'.

The AUC is threshold-free, which is both its strength and its limitation: it says nothing directly about sensitivity and specificity at a particular clinical threshold. Nor does it measure calibration (how well predicted probabilities match observed rates); any monotonic change of the scores leaves the AUC unchanged. With very imbalanced classes, the precision-recall curve adds information. Confidence intervals come from the DeLong, Hanley-McNeil or bootstrap methods, and two correlated AUCs are compared with DeLong's test. An AUC computed on the data used to build the model is optimistic.

An everyday example

A blood test is evaluated for how well it separates people with a disease from those without. When a randomly chosen patient is compared with a randomly chosen healthy person, the patient's test value is higher in 85% of cases. This means AUC = 0.85: the test discriminates well, but not perfectly.

In MerQur

The 'AUC CI method' (DeLong / Bootstrap / Hanley-McNeil) in the ROC Curve; the AUC is also reported in the classification output of Logistic Regression, in the True Skill Statistic and in the performance output of Random Forest, Support Vector Machine and Gradient Boosting.

Analyses where this concept appears

Logistic RegressionROC CurveTrue Skill Statistic (TSS)Random ForestSupport Vector Machine (SVM)Gradient Boosting

Related concepts

DeLong test and AUC confidence intervalSensitivity and specificityPartial AUC