MerQuR+Statistics Glossary

Statistics Glossary → Holm (step-down Bonferroni) method

Holm (step-down Bonferroni) methodconcept

A multiple-comparison method that sorts p values from smallest to largest and compares each with a progressively less strict threshold, controlling the familywise error rate.

For m tests the p values are sorted: p(1) ≤ p(2) ≤ … ≤ p(m). The smallest p is compared with α/m, the second with α/(m - 1), and the i-th with α/(m - i + 1). The procedure stops at the first p that exceeds its threshold, and that test and all later ones are not rejected. The results can also be reported as adjusted p values.

Holm's method keeps the familywise error rate (the probability of at least one false positive) at α just like Bonferroni, but it is always at least as powerful; it makes no assumption about dependence between the tests. It can therefore replace Bonferroni in almost every situation. The Holm-Šidák version is slightly more powerful when the tests are independent or positively dependent. It serves a different goal from the Benjamini-Hochberg method, which controls the false discovery rate.

An everyday example

A doctor checks 5 separate blood values in a patient, and the p values are 0.004, 0.011, 0.02, 0.04 and 0.30. With Holm, 0.004 < 0.05/5 = 0.01 (significant), 0.011 < 0.05/4 = 0.0125 (significant), 0.02 > 0.05/3 ≈ 0.0167 (stop). The first two are significant, whereas Bonferroni would have flagged only the first.

In MerQur

One of the main methods of the Multiple Comparison Correction analysis, whose 'Additional methods' also include Holm-Šidák. It is used as a p-value adjustment in Repeated Measures ANOVA, Kruskal-Wallis, Friedman, Cochran's Q, the chi-square goodness-of-fit test, multiple-response crosstabs, correlation analysis and Kaplan-Meier pairwise comparisons.

Analyses where this concept appears

Repeated Measures ANOVAMultiple Comparison CorrectionKruskal-WallisFriedmanChi-Square Goodness of FitMR × Categorical CrosstabMR × MR CrosstabCochran's Q TestCorrelation AnalysisKaplan-Meier Survival Analysis

Related concepts

Bonferroni correctionMultiple comparisons and the familywise error rate (FWER)