Statistics Glossary → Scheffé test
Scheffé testconcept
A multiple-comparison method used after ANOVA that tests not only pairwise comparisons but all possible linear contrasts of the group means at once, while controlling the familywise error rate.
A contrast is a combination of group means whose coefficients sum to zero; for example, 'does the average of A and B differ from C?' is a contrast. Scheffé's method compares each contrast's F statistic with (k - 1) times the critical F value (k being the number of groups). This threshold keeps the probability of at least one false significant result at α even if infinitely many contrasts are tested. Contrasts remain valid even when they are chosen after looking at the data.
The price of this breadth is conservatism: if only pairwise comparisons are wanted, Tukey's HSD gives narrower intervals and more power. A useful property of Scheffé's test is its exact agreement with the overall F test: if the overall F is not significant, no Scheffé contrast will be; if it is significant, at least one contrast (not necessarily a pairwise one) is. Its assumptions are those of ANOVA (normality, equal variances, independence).
An everyday example
A company tests four different advertising texts. After seeing the results, the marketing team wants to ask whether the average of the two humorous texts beats the average of the two serious ones. Because this question arose from looking at the data, an ordinary t-test would mislead; the Scheffé test keeps the error rate under control even for such after-the-fact comparisons.
In MerQur
It is one of the choices under 'Post-hoc test' in One-Way ANOVA and Two-Way ANOVA, alongside Tukey HSD, Bonferroni and other methods, and gives adjusted p-values for the pairwise differences.
Analyses where this concept appears
Related concepts
Post hoc tests (pairwise comparisons)Contrast and custom contrast (L)