MerQuR+Statistics Glossary

Statistics Glossary → Distribution-free (order-statistic) confidence interval

Distribution-free (order-statistic) confidence intervalconcept

A confidence interval for quantities such as the median or other percentiles that sorts the data and uses observations at particular ranks as its limits, requiring no distributional assumption such as normality.

Once the data are sorted, the probability that the population median lies between two order statistics can be computed from the binomial distribution regardless of the distribution the data came from, since each observation falls below the median with probability 0.5. For example, with 20 observations the interval between the 6th and 15th ordered values covers the median with probability of about 95.9%. The same reasoning applies to other percentiles; the non-parametric limits of agreement in a Bland-Altman analysis rest on the 2.5th and 97.5th percentiles of the differences.

The advantage of these intervals is that they assume only that observations are independent and come from the same distribution; they hold up well with skewed data and outliers. Because of their discreteness the coverage cannot be set to exactly 95%; it usually lands a little above, or interpolation between two ranks is used. When the data really are normal they are slightly wider than parametric intervals. Reliable results for extreme percentiles (such as the 2.5th) require fairly large samples.

An everyday example

The electricity use of 20 homes in a neighbourhood is sorted, and a few very heavy users skew the distribution to the right. The median is 210 kWh. If the 6th value in the sorted list is 180 kWh and the 15th is 250 kWh, then 180-250 kWh is a roughly 96% confidence interval for the neighbourhood's median consumption that does not assume normality.

In MerQur

The Sign Test gives a distribution-free confidence interval for the median difference built from order statistics (the 'Confidence level' option sets the requested level; the attained coverage can differ from it). In Bland-Altman Agreement Analysis, 'Additional outputs: Non-parametric LoA' gives the 2.5th, 50th and 97.5th percentiles of the differences. The Permutation Test is used as a distribution-free alternative to the t test and ANOVA.

Analyses where this concept appears

Permutation TestSign TestBland-Altman Agreement Analysis

Related concepts

Hodges-Lehmann estimator (pseudomedian)Percentiles (quantiles) and percentile definitions