MerQuR+Statistics Glossary

Statistics Glossary → Precision-recall curve and average precision

Precision-recall curve and average precisionconcept

A curve showing a classifier's precision (how many positive predictions are correct) against its recall (how many true positives are caught) across thresholds, and average precision (AP), which summarises it in a single number.

Precision is TP / (TP + FP) and recall (sensitivity) is TP / (TP + FN). As the threshold is lowered, recall rises and precision usually falls; the PR curve plots this trade-off over all thresholds. Average precision summarises the area under the curve as a step sum: AP = Σ (Rₙ − Rₙ₋₁)·Pₙ. Because precision does not change linearly between successive points, joining the points with straight lines and computing the area gives an optimistic result.

Because the ROC curve also uses true negatives, when the positive class is very rare a high AUC can hide a large number of false positives that make the classifier useless in practice. The PR curve focuses directly on the positive class and is therefore more informative for such imbalanced problems. The PR curve of a random classifier is a horizontal line at the prevalence, so AP should always be interpreted together with the prevalence, and AP values cannot be compared directly across data sets with different prevalences.

An everyday example

An email service develops a filter where only 2% of messages contain malware. The ROC AUC is 0.95. But the PR curve shows that at the threshold chosen to catch 80% of malware, only 3 of every 10 flagged messages are actually harmful. The AP is 0.42, whereas a random filter would have an AP of 0.02.

In MerQur

In ROC curve analysis, the 'Additional output: PR curve and average precision (AP)' option gives the precision-recall curve and the step-sum average precision.

Analyses where this concept appears

ROC Curve

Related concepts

Area under the ROC curve (AUC)F1 score, MCC and balanced accuracy