Statistics Glossary → McNemar Test
McNemar Testanalysis
A test of whether the proportion of a yes/no outcome measured twice on the same people changes between the two measurements, based only on those whose answer or status changed.
Concept
In before-after studies, or when two diagnostic methods are applied to the same cases, the observations are not independent: each person is measured twice. The ordinary chi-square test therefore cannot be used. The McNemar test asks whether the proportion of 'yes' changes from the first measurement to the second while taking this pairing into account.
The paired data are summarised in a 2x2 table. People who give the same answer both times (yes-yes, no-no) carry no information about change; the test looks only at the discordant pairs: those who moved from yes to no (b) and from no to yes (c). If there is no real change, moves in the two directions should be equally frequent. The test statistic is χ² = (b − c)² / (b + c); when the number of discordant pairs is small, an exact test checks whether b follows a binomial distribution within b + c.
Because the result rests on the number of discordant pairs, the test has little power when few people change, even in a large sample. The size of the change should be reported as the difference between the two proportions or as the conditional odds ratio (the ratio of the two directions of change). With more than two categories, the McNemar-Bowker test of symmetry is used.
When to use it
- When comparing a binary outcome measured twice on the same people (e.g. before and after training, two diagnostic tests applied to the same patients, matched case-control pairs).
- When two independent groups are compared, use the chi-square test of independence or Fisher's exact test.
- With three or more binary measurements on the same people use Cochran's Q test; if the outcome is continuous, the paired t-test or the Wilcoxon signed-rank test is appropriate.
An everyday example
A company asks its 200 employees 'Do you eat breakfast regularly?' before and after a healthy-eating campaign. Most give the same answer both times; 30 switch from 'no' to 'yes' and 10 from 'yes' to 'no'. If the campaign had no effect, these 40 switchers would be expected to split roughly evenly between the two directions. The McNemar test gives the probability that a 30-to-10 split would arise by chance (exact p ≈ 0.002), showing that the share of employees who eat breakfast rose after the campaign.
Parameters in MerQur
Before / 1st measurement and After / 2nd measurement (two binary columns)
Concept: Two columns holding the two measurements of the same person; each row is one person, and the pairing comes from this row structure. The order of the columns does not change the test statistic, only the direction in which the change is expressed (increase or decrease).
Everyday example: In the company example the first column holds the answer before the campaign and the second the answer after it. Swapping them leaves p unchanged, but the difference in proportions appears with the opposite sign, as a 'decrease' rather than an 'increase'.
What it does: Defines the two paired measurements. b is the number moving from the first level to the second (n12); c the reverse transitions (n21). With more than two levels the McNemar-Bowker test runs. A level not observed in one measurement is completed from the union of the two columns.
Effect on the result: Swapping the columns does not change χ² and p; b/c, OR (b/c), Cohen g, and the sign of the proportion difference change.
On the validation data: In Performance, swapping the columns keeps χ² the same; OR goes from 1.7442 to 0.5733, the proportion difference from +0.04 to -0.04. For a measurement with a single observed level, b = 10, c = 0 gives exact p = .001953.
Method (Automatic, Exact (binomial), Asymptotic (χ², Yates), Asymptotic (χ², no correction))
Concept: Sets how the p value is computed. The exact (binomial) method computes the distribution of discordant pairs directly and is safe when there are few of them; the asymptotic method uses the chi-square approximation, and the Yates correction makes that approximation more cautious.
Everyday example: Had only 8 employees changed their answer, the chi-square approximation would be rough and the exact binomial test would be preferred. With 40 switchers the methods give similar results.
What it does: The automatic option uses the exact binomial test if the number of discordant pairs (n_disc) is below 25, otherwise the Yates-corrected asymptotic test. The exact method is safe in small samples; the Yates correction is more conservative.
Effect on the result: The test statistic and p change; b, c, and the proportion difference do not. In k x k tables the method choice is not used (the Bowker test runs) and a note is written on screen.
On the validation data: Performance: Yates χ² = 16.8178, uncorrected 17.3559 (p = 3.10e-05), exact p = 3.72e-05. Q1/Q2 (n_disc = 13): Automatic exact p = 1.
Confidence Level (%90, %95, %99, Other (custom))
Concept: Sets the width of the confidence intervals for the difference in proportions between the two measurements and for the conditional odds ratio. The test statistic and p value do not change.
Everyday example: If the share of employees eating breakfast rose by 10 percentage points, the 95% confidence interval might say the rise lies somewhere between 4 and 16 points; at 99% the interval widens.
What it does: Sets the confidence level of the Wald interval for the proportion difference (b - c)/n, the Agresti-Min/Tango intervals, and the conditional OR interval. The custom value is entered in the range 80.0-99.9.
Effect on the result: Only the confidence limits change; the test statistic and p do not.
On the validation data: Performance Wald interval: 90% [.0243, .0557], 95% [.0213, .0587], 99% [.0154, .0646]; custom 80.0 [.0278, .0522], 99.9 [.0086, .0714].
Hypothesis direction (Two-sided (≠), Right-tailed (>), Left-tailed (<))
Concept: States whether the research question looks for any change (two-sided) or for change in a pre-specified direction. The direction should be chosen from the research question, not from the data.
Everyday example: If the company only wants to know whether the campaign increased breakfast eating, it may choose a one-sided test. If a backfiring campaign is also a concern, the two-sided test is appropriate.
What it does: Performs a one-sided test if the direction is known in advance (right: b > c).
Effect on the result: When a direction is selected, the main p and the decision become one-sided (binomial in the exact method, signed z in the asymptotic one); the two-sided p is written separately.
On the validation data: Performance one-sided exact p (>) = 1.86e-05, left-tailed .99999; p = 2.057e-05 for the asymptotic signed z. Q1/Q2 left-tailed exact p = .5; Drugs one-sided exact p = .000656.
Extra outputs: mid-p
Concept: A correction that subtracts half the probability of the observed outcome from the exact binomial p value. It reduces the excess conservatism of the exact test that comes from whole-number counts.
Everyday example: With few switchers, say 9 to 2, the exact p may sit just above the threshold while the mid-p falls below it. Reporting both then makes clear that the finding is borderline.
What it does: Subtracts half the probability of the observed point from the exact binomial p (mid-p); reduces the conservatism of the exact test.
Effect on the result: Adds only an extra p row; the main results do not change.
On the validation data: Performance 2.89e-05; Q1/Q2 .7905; right-tailed 1.40e-05.
Extra outputs: Agresti-Min / Tango CI
Concept: Confidence intervals for the difference between two paired proportions that behave better than the simple Wald interval; in small samples, or when proportions are near 0 or 1, the Wald interval may fail to cover the true value as often as it should.
Everyday example: Had the company surveyed only 30 employees, the Wald interval for the difference could come out misleadingly narrow or shifted. The Agresti-Min and Tango intervals give a more trustworthy margin of uncertainty in samples this small.
What it does: Gives the adjusted Wald (Agresti and Min, 2005) and score (Tango, 1998) confidence intervals for the paired proportion difference.
Effect on the result: Extra interval rows only.
On the validation data: Performance Agresti-Min [.02122, .05868], Tango [.02136, .05900]; Drugs lower limits Agresti-Min .2360, Tango .2263.
→ Confidence interval for a proportion (Wilson, Clopper-Pearson, Agresti-Coull, Jeffreys)
Extra outputs: Conditional OR CI
Concept: The conditional odds ratio is the ratio of the moves in the two directions (e.g. b/c). It shows how many times more frequent one direction is than the other among those who changed; 1 corresponds to no change.
Everyday example: In the company example the 30 people who switched from 'no' to 'yes' are 3 times the 10 who switched the other way. The exact confidence interval runs from about 1.4 to 6.9; since it excludes 1, the direction of change is reliable, though its size is still uncertain.
What it does: Gives the conditional OR = b/c and the exact confidence interval based on Clopper-Pearson.
Effect on the result: Extra OR rows only. No effect in k x k tables, and a note is written.
On the validation data: Performance OR 1.7442 [1.3292, 2.3010]; 99% [1.2254, 2.5080]; Q1/Q2 .8571 [.2380, 2.9786].
Example on real data
Data: Performance (prime minister approval), Agresti 1990, R ?mcnemar.test
Variables: before Survey1 (approval in the 1st survey), after Survey2 (approval of the same 1600 people in the 2nd survey)
- Question: Did the approval of the prime minister by the same 1600 people change between the two surveys? The McNemar test, which uses only those who changed their opinion in paired binary data, is applied.
- Descriptive: 150 people moved from approve to disapprove (b) and 86 from disapprove to approve (c); there are 236 discordant pairs.
- Result: Yates-corrected McNemar χ²(1) = 16.82, p < .001. The null hypothesis is rejected: the approval rate changed significantly between the two surveys.
- Effect size: Proportion difference 0.040, 95% CI [0.021, 0.059]; the approval rate fell by about 4 points in the second survey. OR_disc = 1.74, Cohen g = 0.136 (small effect).
- Note: The McNemar statistic can be given with the Yates correction (16.82) or without it (17.36); the decision is the same, and the report should state which form is used.
- APA: McNemar χ²(1, N = 1600) = 16.82 (Yates-corrected), p < .001, g = .14.
Related concepts
Wald test and Wald confidence intervalAsymptotic (normal) approximationMid-p valueConfidence interval for a proportion (Wilson, Clopper-Pearson, Agresti-Coull, Jeffreys)Odds ratio (OR)Exact test and exact p-valueOne-sided and two-sided tests (hypothesis direction)Continuity correction (Yates)
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.