Statistics Glossary → Kruskal-Wallis
Kruskal-Wallisanalysis
A test that needs no normality: it ranks the values of three or more independent groups together and compares the groups' mean ranks, testing whether at least one group differs systematically from the others.
Concept
One-way ANOVA compares the means of three or more groups and assumes normality and equal variances within each group. When the data are skewed, contain marked outliers, the groups are small or the measure is ordinal, these assumptions weaken. The Kruskal-Wallis test is the extension of the Mann-Whitney U test to three or more groups: it uses ranks instead of values and needs no assumption about the shape of the distribution.
All observations from all groups are ranked together from smallest to largest; tied values receive the average rank. Each group's mean rank is then computed. If there is no difference between the groups, every group's mean rank should be close to the overall mean rank, (N + 1)/2. The H statistic summarises how far the groups' mean ranks lie from this value and, in large samples, approximately follows a chi-square distribution with k − 1 degrees of freedom. The larger H, the smaller p.
Observations are assumed to be independent. The hypothesis tested is that no group's values tend systematically to be larger or smaller than another's; reading the result as 'the medians differ' is correct only when the groups have similar distribution shapes and spreads. A significant H says only that at least one group differs; which groups differ is shown by post-hoc comparisons such as Dunn's test. The effect size (η²_H or ε²) shows the practical importance of the difference.
When to use it
- When comparing three or more independent groups on a continuous or ordinal measure, especially with small groups, skewed distributions, marked outliers, or ordinal-scale data.
- With only two groups, use the Mann-Whitney U test; if the same people were measured under every condition, use the Friedman test.
- If the data are near-normal within each group and the groups large enough, one-way ANOVA (Welch's ANOVA if variances differ) is more powerful; if the groups have a natural order and a steady rise or fall is expected, the Jonckheere-Terpstra test is more suitable.
An everyday example
A municipality wants to compare how long passengers wait for the bus at stops in the city centre, the middle districts and the outskirts, and records 20 waiting times from each area. Waits are mostly 5-15 minutes, but a few passengers on the outskirts waited more than 45 minutes; the distribution is skewed. The Kruskal-Wallis test ranks all 60 waits and compares the three areas' mean ranks. If the mean ranks are 20.5 in the centre, 30 in the middle districts and 41 on the outskirts with p = 0.001, the waiting times in at least one area differ from the others; Dunn's test may show that the difference lies mainly between the centre and the outskirts.
Parameters in MerQur
Post-hoc test (None, Dunn (Bonferroni), Dunn (Holm), Dunn (Benjamini-Hochberg), Dunn (Hochberg), Dunn (Šidák), Conover-Iman, Nemenyi, Dwass-Steel-Critchlow-Fligner)
Concept: When the Kruskal-Wallis test is significant, post-hoc tests show which pairs of groups differ while controlling the false-alarm risk of multiple comparisons. Dunn's test uses the joint ranking of all groups and can be combined with different p adjustments (Bonferroni, Holm, Benjamini-Hochberg…); Conover-Iman is usually more powerful; Nemenyi and Dwass-Steel-Critchlow-Fligner are Tukey-type methods that treat all pairs as a single family.
Everyday example: Three areas form three pairs. Dunn's test with a Holm adjustment might conclude that 'the centre and the outskirts differ, while the middle districts do not differ significantly from either'; the municipality then sends extra buses to the outskirts first.
What it does: If KW is significant, tests which group pairs differ. The Dunn options use the same tie-corrected z; only the multiple comparison adjustment changes. Conover-Iman is based on the t distribution; Nemenyi is Tukey-type without tie correction; DSCF tests each pair with its own two-sample ranking.
Effect on the result: Adds only the post-hoc section; H, p, and the effect size do not change.
On the validation data: hsb2 pair 1-3 (z = 2.150): raw p 0.0316; Bonferroni 0.0947 (not significant), Holm 0.0316, BH 0.0316, Šidák 0.0917. For the same pair Conover-Iman p = 0.0515, Nemenyi 0.0811, DSCF 0.0516.
Extra output: ε² (checkbox; default off)
Concept: The effect size shows how much of the variability in the ranks comes from the group difference. η²_H and ε² are rank-based counterparts of η² in ANOVA; both are read on a 0-to-1 scale and give similar values in large samples.
Everyday example: An ε² of 0.23 means that about 23% of the variation in the ranks of waiting times is related to the area. This shows that where a stop is located plays a sizeable role in how long people wait.
What it does: The rank-based epsilon-squared effect size, ε² = H/(N - 1).
Effect on the result: Adds ε² next to η²_H in the effect size row; the test does not change. η²_H is clipped to 0 when H < k - 1; ε² is always positive (Tomczak and Tomczak, 2014).
On the validation data: hsb2 ε² = 0.1711 (η²_H 0.1627); airquality 0.2545 (0.2276); mucociliary 0.0593 (η²_H 0.0000).
Extra output: Descriptive (checkbox; default off)
Concept: Gives each group's summary: count, mean, standard deviation, standard error, median, minimum and maximum, skewness and kurtosis. Before a rank-based test, it shows whether the groups' distribution shapes are similar and where the outliers are.
Everyday example: The table shows a mean of 18 and a median of 12 minutes on the outskirts; a few very long waits inflate the mean. In the centre the mean and median are close.
What it does: n, mean, SD, SE, median, minimum, maximum, skewness (G1), kurtosis (G2) per group.
Effect on the result: Adds only a descriptive table.
On the validation data: hsb2 group 1: M = 51.3333, SD = 9.3978, median 54, skewness -0.406, kurtosis -0.650. In a constant group skewness and kurtosis are undefined.
Confidence Level (%90, %95, %99, Other (custom), 80.0-99.9)
Concept: The confidence level sets the significance threshold used to reject the null hypothesis (α = 1 − confidence level). It does not change the H statistic or the p value; it only changes which results count as 'significant'.
Everyday example: If p = 0.02, the areas count as different at the 95% level but not at the 99% level. If the municipality is planning a major investment in new routes, it may want a stricter threshold.
What it does: α = 1 - confidence level. KW produces no confidence interval; α determines only the H0 decision and the marks in the post-hoc rows. The Monte Carlo CI is fixed at 99%.
Effect on the result: Only the decision and marks. The custom value is active only when "Other" is selected.
On the validation data: The form value is passed to the engine exactly (90% -> 0.10, custom 80.0 -> 0.20). In the five datasets p is either very small or very large, so the decision does not change; in an additional design (p = 0.0273) the result is "rejected" at 95% and "not rejected" at 99%.
Extra outputs: Detailed pairwise table (+ r) (checkbox)
Concept: The detailed pairwise table shows, for each pair of groups, the mean rank difference, its standard error, the z value, the raw and adjusted p values, and the effect size r = |z|/√N. It shows not only which pair differs but also how large the difference is.
Everyday example: The mean rank difference between the centre and the outskirts might be 20.5, with an adjusted p below 0.001 and r = 0.48; between the centre and the middle districts the difference might be 9.5 with an adjusted p of 0.17. The municipality reads from this table which difference really matters.
What it does: Tabulates the Dunn comparisons with ΔR̄, SE, z, raw p, adjusted p, and r = |z|/√N. The adjustment is the Dunn adjustment selected in the post-hoc box; otherwise Bonferroni.
Effect on the result: Adds only the pairwise table.
On the validation data: hsb2 1-2: ΔR̄ = -30.917, SE = 10.286, z = -3.006, p = 0.0026, Bonferroni 0.0079, r = 0.213; 2-3: z = 5.688, r = 0.402.
Extra outputs: Jonckheere-Terpstra (checkbox)
Concept: If the groups have a natural order (dose levels, education stages, distance from the centre), the Jonckheere-Terpstra test checks whether values rise or fall steadily along this order. When such a trend is predicted in advance, it is more powerful than the Kruskal-Wallis test, which asks only 'is there any difference?'. It is not meaningful when the groups are unordered categories (e.g. colours, cities).
Everyday example: The areas are ordered by distance from the centre: centre, middle districts, outskirts. If the municipality expects that 'waits get longer the further you go from the centre', Jonckheere-Terpstra tests exactly this steady increase.
What it does: The J = Σ U_ij test against the alternative that the groups change monotonically in an ordered sequence (natural order; ascending if numeric, otherwise alphabetical) (tie-corrected normal approximation, two-sided p).
Effect on the result: Extra section only. Because the order is derived from the names, it is not interpretable for nominally named groups.
On the validation data: hsb2 (1 < 2 < 3) J = 5100, E(J) = 6112.5, SD = 429.0865, z = -2.3597, p = 0.0183; gossypol z = -7.7715.
Extra outputs: Monte Carlo p (checkbox)
Concept: The chi-square approximation to the H statistic can weaken with small groups or many equal values (ties). The Monte Carlo p value generates the null distribution of H from the data by shuffling the group labels thousands of times, giving a p that does not rely on this approximation.
Everyday example: Because waits are rounded to the whole minute, many passengers have the same time. The 60 waiting times are kept fixed while the area labels are shuffled at random and H is recomputed each time; the share of values at least as large as the observed H is the Monte Carlo p.
What it does: Produces p from the permutation distribution instead of the asymptotic χ² (B = 10000, seed 12345; 99% Clopper-Pearson CI). p = (b + 1)/(B + 1) (Phipson and Smyth, 2010).
Effect on the result: Extra p row only. In small or heavily tied samples it can differ markedly from the asymptotic p.
On the validation data: mucociliary 0.7068 [0.6949, 0.7185] (asymptotic 0.680). In hsb2 b = 0; MerQur gives 1/10001 = 0.0001.
Extra outputs: Median test (checkbox)
Concept: Mood's median test classifies each observation only as above or below the overall median, and compares how the groups are split between these two classes with a chi-square test. Because it does not use the detail of the ranks it is less powerful than Kruskal-Wallis, but it is completely unaffected by outliers.
Everyday example: Let the overall median wait across all areas be 10 minutes. If only 30% of passengers in the centre but 70% on the outskirts wait more than 10 minutes, the median test tests this difference; for this test a 45-minute wait is no different from an 11-minute one.
What it does: Mood's k-sample median test: Pearson χ² in the 2 x k table of above and at-or-below the grand median (no Yates correction).
Effect on the result: Extra section only; less powerful than KW.
On the validation data: hsb2 grand median 54, χ²(2) = 23.4562, p = 8.06e-06; gossypol χ²(4) = 54.9973.
Score / scale tests: Van der Waerden + Savage; Scale tests (Siegel-Tukey, Ansari-Bradley, Mood, Klotz, Fligner-Killeen) (checkboxes)
Concept: Instead of the ranks themselves, other scores derived from the ranks can be used. The Van der Waerden test converts ranks to normal scores and is as powerful as ANOVA when the data are near-normal; the Savage test is powerful for exponential-type data such as waiting times or lifetimes. The scale tests (Siegel-Tukey, Ansari-Bradley, Mood, Klotz, Fligner-Killeen) test not the groups' location but whether their spreads differ.
Everyday example: If the municipality suspects that waits on the outskirts are not only longer but also more 'unpredictable', it looks at the scale tests. If Fligner-Killeen is significant, the spread of waiting times differs between areas, which may suggest that the timetable is not being kept.
What it does: Location (VW, Savage) and scale (Siegel-Tukey, Ansari-Bradley, Mood, Klotz) tests in k samples with score definitions that use the mean score for ties; Fligner-Killeen separately. Chi-square and p are computed with k - 1 degrees of freedom.
Effect on the result: Added as an extra section at the end of the Kruskal-Wallis output.
On the validation data: For chickwts (weight ~ feed, k = 6), chi-square and p are given with 5 degrees of freedom. hsb2 write ~ prog scale tests: Siegel-Tukey χ²(2) = 2.7818, p = 0.2489; Ansari-Bradley 2.7813, p = 0.2489; Mood 2.3449, p = 0.3096; Klotz 2.2456, p = 0.3254; Fligner-Killeen 3.2550, p = 0.1964.
→ Score and scale tests (Van der Waerden, Savage, Siegel-Tukey, Ansari-Bradley, Mood, Klotz)
Example on real data
Data: UCLA OARC hsb2 (High School and Beyond, n = 200)
Variables: group prog (programme type, 3 levels: 1 general, 2 academic, 3 vocational), value write (writing score)
- Question: Does the writing score (write) differ among the three programme types (general, academic, vocational)? The Kruskal-Wallis test, the rank-based counterpart of one-way ANOVA, is used.
- Descriptive: Mean ranks are 121.56 in the academic programme (median 59), 90.64 in the general programme (median 54), and 65.14 in the vocational programme (median 46).
- Result: H(2) = 34.05, p = 4.05e-08. The null hypothesis is rejected: the score distribution of at least one programme differs from the others.
- Effect size: η²_H = 0.16; about 16% of the variability in the ranks is explained by programme type (large effect).
- Next step: To see which programmes differ, the pairwise comparison option (e.g., Dunn's test with Bonferroni or Holm adjustment) should be used.
- APA: H(2) = 34.05, p < .001, η² = .16.
Related concepts
Monte Carlo p-valuePost hoc tests (pairwise comparisons)Jonckheere-Terpstra test (ordered alternative)Mood's median testEpsilon squared (ε²)Score and scale tests (Van der Waerden, Savage, Siegel-Tukey, Ansari-Bradley, Mood, Klotz)Ties and zero differences (rank tests)Eta squared (η²) and partial eta squared (η²ₚ)
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.