MerQuR+Statistics Glossary

Statistics Glossary → DeLong test and AUC confidence interval

DeLong test and AUC confidence intervalconcept

A method that computes the standard error of the area under the ROC curve (AUC) nonparametrically and compares two AUCs measured on the same subjects.

The AUC is the probability that a randomly chosen positive case has a higher score than a randomly chosen negative case; it is the same quantity as the Mann-Whitney U statistic. DeLong, DeLong and Clarke-Pearson (1988) use this U-statistic structure to compute the variance of the AUC without distributional assumptions. That variance gives a confidence interval for the AUC and a test of whether it differs from 0.5 (random discrimination).

When two tests or models are applied to the same subjects, their AUCs are correlated, and comparing them as if they were independent is wrong. The DeLong test accounts for this covariance and makes a paired comparison. A non-significant difference does not show that the two tests are equivalent. Small differences in AUC may be clinically unimportant, while performance at the chosen threshold may differ. When comparing nested models (adding a variable to a model), the DeLong test is known to be overly conservative, and a likelihood ratio test may be preferred.

An everyday example

In a hospital, 200 patients receive two different blood tests, and the ability of each test to identify a disease is measured. The first test has an AUC of 0.82, the second 0.76. Because both tests were done on the same patients, the paired DeLong test is used; if p = 0.04, the first test discriminates significantly better.

In MerQur

In ROC curve analysis, setting the 'Multiple variables' option to 'Separate curve per test variable (+ paired DeLong)' compares each pair of curves with a paired DeLong test, and the 'AUC CI method' can be DeLong, Bootstrap (stratified, 2000) or Hanley-McNeil. In the true skill statistic, the 'TSS CI + Kappa + AUC' option gives the AUC by the DeLong method.

Analyses where this concept appears

ROC CurveTrue Skill Statistic (TSS)

Related concepts

Area under the ROC curve (AUC)