MerQuR+Statistics Glossary

Statistics Glossary → One-Way ANOVA

One-Way ANOVAanalysis

An analysis of variance that tests whether the means of three or more independent groups are equal, by comparing the differences between the groups with the spread within them.

Concept

When comparing three or more groups, running a separate t-test for every pair is misleading: each test carries a false-alarm risk, and as the number of tests grows, the chance of at least one false 'there is a difference' rises quickly. One-way ANOVA handles all the groups in a single test of the null hypothesis that all group means are equal. A significant result says only that at least one group differs from the others; which groups differ is shown by post-hoc tests or by contrasts planned in advance.

The method gets its name from its logic: the total variability is split in two. The between-group variability shows how far the group means lie from the overall mean; the within-group variability shows how much observations in the same group scatter around their own mean. F = between-group mean square / within-group mean square. If the groups are truly the same, F comes out around 1; as the group means move apart, F grows and p shrinks. Effect sizes such as η² show how much of the total variability comes from the differences between groups.

Classical ANOVA assumes independent observations, an approximately normal outcome within each group and equal group variances. With similar group sizes, moderate departures from normality and from equal variances do little harm; but when variances differ and group sizes are unbalanced, Welch's ANOVA should be preferred. Significance alone is not enough: the size of the difference and the groups it comes from should also be reported.

When to use it

  • When comparing a continuous outcome across independent groups formed by three or more levels of a single factor (e.g. three fertiliser doses, four teaching methods).
  • If group variances clearly differ, use Welch's ANOVA; with strongly skewed distributions and small groups, use the Kruskal-Wallis test. With only two groups, the independent-samples t-test is enough.
  • If the same subjects are measured under every condition, use repeated-measures ANOVA; if two or more factors are studied together, a two-way (factorial) ANOVA is needed.

An everyday example

A farm wants to see how fertiliser dose affects tomato yield, so it divides 30 plots into three groups of 10: no fertiliser, low dose and high dose. The mean yield per plant comes out at 4.7, 5.0 and 5.5 kg respectively. One-way ANOVA compares the differences between these three means with how spread out the yields are among plots in the same group. If F(2, 27) = 4.9 and p = 0.016, at least one dose group's mean differs from the others; post-hoc tests show which doses differ from one another.

Parameters in MerQur

Post-hoc test (None, Tukey HSD, Duncan MRT, Bonferroni, Scheffé, Games-Howell, LSD, Šidák, Hochberg GT2, Gabriel, Student-Newman-Keuls, Ryan-Einot-Gabriel-Welsch Q, Dunnett (vs. control), Tamhane T2, Dunnett T3)

Concept: After a significant ANOVA, post-hoc tests make all pairwise comparisons to find which pairs of groups differ, while keeping the familywise error rate, the chance of at least one false alarm among these comparisons, under control. The methods differ in how strict this control is and in their assumptions: Tukey's HSD is used for all pairs with equal variances, Games-Howell for unequal variances, Dunnett for comparisons against a control group only, and Bonferroni and Scheffé for more conservative adjustments; LSD makes no adjustment at all.

Everyday example: Three dose groups form three pairs. If each pair is tested at an unadjusted 5% risk, the overall false-alarm risk can approach 14%. Tukey's HSD keeps this risk at 5% and might conclude that 'high dose differs from no fertiliser, while low dose differs from neither'.

What it does: After the ANOVA, shows which pairs of groups differ while controlling the familywise error rate. REGWQ is a step-down method: α_r = 1 - (1 - α)^(r/k) (r < k - 1), α_r = α for r >= k - 1.

Effect on the result: F, p and SS do not change; per pair the difference, the adjusted p, the simultaneous CI and a significance flag are added. The selected method runs even if the ANOVA is not significant.

On the validation data: For the PlantGrowth trt1 - trt2 difference (-0.865), p: LSD 0.0045, Tukey HSD 0.0120 ([-1.556, -0.174]), Bonferroni 0.0134, Scheffé 0.0163, Games-Howell 0.0237, Tamhane T2 0.0276. In the recall data, Tukey p = .011, Games-Howell p = .056.

→ Post hoc tests (pairwise comparisons)

Confidence Level (%90, %95, %99, Other (custom), 80.0-99.9)

Concept: The confidence level sets how 'sure' the intervals for differences and means are, and the threshold (α) for deciding that a difference exists. It does not change the F value itself, only what counts as significant.

Everyday example: At the 95% confidence level p = 0.016 is significant; at the 99% level the same result counts as 'not significant'. If a strong claim is to be reported to a fertiliser company, a higher confidence level may be required.

What it does: α = 1 - confidence level; sets the decision threshold and the width of all CIs.

Effect on the result: F, p and SS do not change. The H0 decision, the post-hoc flags and CIs, the effect size CIs, the "met / VIOLATED" wording of the assumption rows and the Mean ± CI chart change.

On the validation data: PlantGrowth Tukey trt1 - trt2 CI 90% [-1.462, -0.268], 95% [-1.556, -0.174], 99% [-1.751, 0.021]; at 99% the pair is flagged not significant. η² CI 95% [0.0110, 0.4902].

→ Confidence level and confidence interval

ANOVA variant (Classic (Fisher), Welch (unequal variances))

Concept: Classical (Fisher) ANOVA assumes equal group variances and uses a single common within-group variance. Welch's ANOVA uses each group's own variance; when variances differ and group sizes are unbalanced, it keeps the false-alarm rate at the correct level.

Everyday example: If high-dose plots yield anywhere from 3 to 8 kg while unfertilised plots cluster between 4 and 5 kg, the variances clearly differ. In that case Welch's ANOVA gives the more reliable result.

What it does: The Welch (1951) F does not assume equal variances.

Effect on the result: The statistic, p and denominator df come from Welch (fractional df2); the SS table remains classic. Welch is undefined for a group with zero variance; MerQur warns and reports the classic F.

On the validation data: For upsit, Welch F(4, 78.75) = 13.72 instead of classic F(4, 175) = 16.65; for recall 14.259, df2 7.211, p 0.0031; sleep F(1, 17.78) = 3.4626.

→ Welch correction

Effect sizes: ω², ε², Effect CIs (checkboxes)

Concept: An effect size shows how much of the total variability comes from the differences between groups. η² tends to overstate this share in a sample; ω² and ε² estimate the population value with less bias and are more realistic, especially in small samples.

Everyday example: With 30 plots, η² = 0.26 says that about a quarter of the variation in yield is explained by fertiliser dose. ω² might give a more cautious value such as 0.20; with 300 plots the two measures would be almost the same.

What it does: η² is biased upward in the sample; ε² (Kelley) and ω² (Hays) are less biased. The CIs come from the noncentral F distribution.

Effect on the result: Extra rows only; the test does not change. Negative ε² and ω² are clipped to 0.

On the validation data: PlantGrowth η² = 0.2641, ε² = 0.2096, ω² = 0.2041; 95% CI η² [0.0110, 0.4902]. recall ε² = 0.5064, ω² = 0.4892; recall η² CI [0.1060, 0.7754].

→ Omega squared (ω²)

Extra output: Brown-Forsythe F* (checkbox)

Concept: The Brown-Forsythe F* is another test of equal means that does not assume equal variances. Read together with Welch's test, it shows how much unequal variances affect the result.

Everyday example: If classical F, Welch and Brown-Forsythe all say 'the doses differ', the decision does not hinge on the variance assumption. If one is significant and the others are not, the result is not robust.

What it does: The Brown and Forsythe (1974) test of equality of means; it does not assume equal variances.

Effect on the result: Extra row: F*, Satterthwaite df2, p. In the classic variant a Welch F row is also added.

On the validation data: PlantGrowth F*(2, 22.21) = 4.8461 (in balanced data F* = F, only df2 changes); upsit F*(4, 133.17) = 18.62; recall F*(2, 7.01) = 8.18, p = 0.0147.

→ Brown-Forsythe F* test

Extra output: Letter display (homogeneous subsets) (checkbox)

Concept: The letter display summarises post-hoc results compactly: groups sharing a letter do not differ significantly, while groups with no letter in common do.

Everyday example: If the table shows 'a' for no fertiliser, 'ab' for low dose and 'b' for high dose, high dose differs from no fertiliser, while low dose shares a letter with both and so differs from neither. This display is very common in agricultural reports.

What it does: Produces a compact letter display from the significant pairs of the selected post-hoc (same letter = no difference).

Effect on the result: Runs only when a post-hoc is selected; not produced for Dunnett vs. control or without a post-hoc.

On the validation data: clover Tukey: 3DOK1 a, 3DOK5 ab, 3DOK7 bc, COMPOS bc, 3DOK4 c, 3DOK13 c. recall Tukey subsets {1,3}, {2}.

→ Compact letter display (homogeneous subsets)

Chart (Box plot, Mean ± CI, Normal Q-Q (per group))

Concept: Plots make the groups' distributions and means visible before the numbers. The box plot shows the median, spread and outliers; the mean ± confidence interval plot shows the means and their uncertainty; a Q-Q plot per group shows each group's normality.

Everyday example: The box plot may show that the high-dose box is longer and that one plot yielded very little; perhaps an irrigation fault affected it. The mean ± CI plot presents the three dose means with their intervals in a single picture.

What it does: Selects the type of the Chart tab. "Mean ± CI" uses each group's own SD and t(n - 1).

Effect on the result: The numeric result does not change; the CI depends on the confidence level.

On the validation data: PlantGrowth ctrl 5.032 [4.615, 5.449].

→ Box plot

Assumption checks: Levene (default on), Bartlett, Shapiro

Concept: Checks the ANOVA assumptions: Levene's and Bartlett's tests check whether group variances are equal, and Shapiro-Wilk checks normality within each group. Bartlett's test is powerful for normal data but sensitive to non-normality; Levene's is more robust.

Everyday example: If Levene's test gives p = 0.30, the spread of yields can be taken as similar across the three doses and classical ANOVA is appropriate. Had it given p = 0.01, switching to Welch's ANOVA would be the better choice.

What it does: Tests homogeneity of variance (mean- and median-centred Levene; Bartlett) and normality per group.

Effect on the result: Information rows only; the ANOVA does not change.

On the validation data: InsectSprays Levene W = 6.4554, p = 6.1e-05; median-centred W = 3.8214, p = 0.0042; Bartlett K² = 25.9598, p = 9.09e-05. PlantGrowth Shapiro trt2 W = 0.941, p = 0.564.

→ Levene's test

Descriptives (checkbox; default on)

Concept: Gives a summary of each group: number of units, mean, standard deviation, standard error, median, minimum and maximum, skewness and kurtosis. It shows the overall picture of the groups before the ANOVA result is interpreted.

Everyday example: The table shows at a glance that each dose has 10 plots, that the means are 4.7, 5.0 and 5.5 kg and that the standard deviations are similar. This is a first sign that classical ANOVA is appropriate.

What it does: Gives n, mean, SD, SE, median, minimum, maximum, skewness (G1) and kurtosis (G2) per group.

Effect on the result: Extra table only.

On the validation data: PlantGrowth: ctrl M = 5.0320, SD = 0.5831; trt1 M = 4.6610, SD = 0.7937; trt2 M = 5.5260, SD = 0.4426. Group means and SDs are also given for hsb2 prog, hsb2 ses, crf24, clover and recall.

Dunnett control group (First (first sorted level), Last (last sorted level))

Concept: Dunnett's test compares each treatment group with a single control group rather than comparing all pairs. Because it makes fewer comparisons, it is more powerful than all-pairs tests with the same error control, so the control group must be identified correctly.

Everyday example: Here the control group is the unfertilised plots. Dunnett answers only 'does low dose differ from no fertiliser?' and 'does high dose differ from no fertiliser?'; it does not compare the two doses with each other.

What it does: Makes the first or last level in natural order (numeric labels numerically, others alphabetically) the control.

Effect on the result: Effective only when Dunnett is selected.

On the validation data: In PlantGrowth with the last level (trt2) as control, trt1 - trt2 p = 0.0085 (significant); with the first level (ctrl) both pairs are not significant. hsb2 prog last (3 = vocation): 1-3 p = 0.0199.

→ Dunnett's test (comparison with a control)

Planned contrasts (coefficient list)

Concept: A planned contrast tests a specific question fixed before looking at the data. The group means are weighted by coefficients that sum to zero; for example, coefficients of -1, 0.5, 0.5 compare the first group with the average of the other two.

Everyday example: If the researcher's real question is 'is fertilising, at whatever dose, better than not fertilising?', the contrast -1, 0.5, 0.5 compares the unfertilised group with the average of the two dose groups in a single test.

What it does: Tests the contrast ψ = Σ c·ȳ in pooled (t, N - k) and Welch (Satterthwaite) versions.

Effect on the result: Contrast rows only. Coefficients that do not sum to 0 trigger a warning; if the number of coefficients is not k, the analysis stops with an error message.

On the validation data: crf24 (Kirk, 1968): (0, 0, 1, -1) ψ = -2.75, SE = 0.605, F = 20.6585; (1, 1, -1, -1) ψ = -9.00, F = 110.634. PlantGrowth (1, -1, 0): ψ = 0.371, t(27) = 1.331, p = 0.194.

→ Contrast and custom contrast (L)

Polynomial trend (None, Linear, Quadratic, Cubic, 4th degree, 5th degree)

Concept: If the groups have a natural order with equal spacing (e.g. doses of 0, 50 and 100 units), polynomial trend tests check how the means change along that order: a linear trend captures a steady rise or fall, a quadratic trend captures curvature that turns at a peak or trough.

Everyday example: If the fertiliser doses are equally spaced, the linear trend answers 'does yield rise steadily as the dose increases?'. Had the quadratic trend been significant, it would suggest that beyond a certain dose the gain in yield levels off or reverses.

What it does: Tests equally spaced orthogonal polynomial contrasts.

Effect on the result: Trend rows only. The degree is clipped to k - 1.

On the validation data: InsectSprays (k = 6): linear F = 5.31 (p = 0.024), quadratic 110.35, cubic 29.38, 4th degree 3.58 (p = 0.063), 5th degree 24.89; hsb2 prog linear F = 6.64, quadratic 34.69.

→ Polynomial (trend) contrasts

Example on real data

Data: Dobson 1983, R datasets PlantGrowth (n = 30)

Variables: group group (ctrl, trt1, trt2), value weight (dried plant weight)

  • Question: Do the three conditions (control, treatment 1, treatment 2) differ in mean dried plant weight?
  • Assumption: Levene W = 1.237, p = 0.306; the variances are taken as homogeneous and the classic (Fisher) ANOVA is appropriate.
  • Result: F(2, 27) = 4.85, p = .016. The null hypothesis is rejected at the 5% level: at least one condition mean differs from the others.
  • Effect size: η² = .26; about a quarter of the variability in weight is due to the condition (a large effect by Cohen's criteria).
  • Descriptive: The highest mean is trt2 (5.53) and the lowest trt1 (4.66); a post-hoc option (e.g. Tukey HSD) shows which pair drives the difference.
  • APA: F(2, 27) = 4.85, p = .016, η² = .26.

Related concepts

Dunnett's test (comparison with a control)Welch correctionBrown-Forsythe F* testCompact letter display (homogeneous subsets)Box plotPolynomial (trend) contrastsContrast and custom contrast (L)Omega squared (ω²)

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.