MerQuR+Statistics Glossary

Statistics Glossary → Bayesian ANOVA

Bayesian ANOVAanalysis

An analysis of variance that judges whether group means differ by comparing models with and without the factors through Bayes factors, giving the strength of evidence for each effect.

Concept

Classical ANOVA gives an F test and p value for each effect; if p is not significant we cannot say 'no effect', and if it is significant we do not know how strong the evidence is. Bayesian ANOVA frames the question as a comparison of models: the 'null' model with only an intercept, single-factor models, the additive model and the model with an interaction. For each model it computes how probable the data are, and the models are compared with Bayes factors.

A model's Bayes factor (BF₁₀) shows how many times more probable the data are under that model than under the null model. The evidence for a particular effect is summarised by the inclusion Bayes factor (BF_incl): how much the data raise the odds of all models containing that effect against all models without it, relative to the prior odds. The sizes of the effects are given a JZS-type prior whose width is set by the scale r. Continuous covariates (Bayesian ANCOVA) can be added to the design, or, with repeated measures, a random effect for each subject.

The assumptions are those of classical ANOVA: independent observations (after accounting for the subject effect with repeated measures), roughly normal distributions within cells and similar variances. Bayes factors are sensitive to the width of the prior, so robustness results for different r values should be read alongside. A small p value is not the same as strong Bayesian evidence; especially for weak effects such as interactions, the two approaches can disagree.

When to use it

  • When comparing the means of three or more groups, or of a two-factor design, and the strength of evidence for each effect (including evidence for 'no effect') is wanted.
  • Use the covariate form (Bayesian ANCOVA) to adjust for a continuous variable such as a baseline measurement; use the repeated-measures form with a subject column when the same subjects were measured under several conditions.
  • With only two groups the Bayesian t-test is enough; if cell variances differ greatly or the distributions are clearly skewed, Welch ANOVA or the Kruskal-Wallis test can be considered in the classical framework.

An everyday example

A farm tests wheat yield with three fertiliser types and two irrigation levels on 36 plots. Bayesian ANOVA compares the fertiliser-only, irrigation-only, additive and interaction models. If the inclusion Bayes factor for fertiliser is 250, the evidence that fertiliser type affects yield is strong; if it is 0.4 for the interaction, the data offer mild evidence that the fertiliser effect does not depend on irrigation.

Parameters in MerQur

Mode (One-way / Two-way), Dependent variable, Factor 1, Factor 2

Concept: A one-way design compares the levels of a single factor; a two-way design examines the main effects of two factors and whether the effect of one changes with the level of the other (interaction). The dependent variable is a continuous measurement.

Everyday example: If only the three fertiliser types are compared, the one-way mode is used; if fertiliser type and irrigation level are studied together, the two-way mode. The dependent variable is wheat yield per plot.

What it does: Selects the one-way or two-way (with interaction) between-groups design and the variables. Factor 2 is active only in the two-way mode.

Effect on the result: Determines the JZS model space: {null, A} in one-way, {null, A, B, A+B, A+B+A×B} in two-way. The classical F table (type II) and the BIC-based BFs are also computed from this design; the BIC BF is the BIC difference of the nested models.

On the validation data: ToothGrowth two-way model BFs: supp 1.199, dose 4.984e12, supp+dose 2.875e14, full model 7.837e14. The BIC BF for supp is 105.03. Decimal-coded dose (0.5, 1, 2) appears in the factor list.

JZS prior scale r (fixed effects)

Concept: The width of the prior on the effect sizes; as r grows, H₁ treats larger group differences as plausible. Because the Bayes factor is sensitive to this value, the result should be checked with several r values.

Everyday example: If small-to-medium differences between fertilisers are expected, the default r = 0.5 is suitable. When the true difference is small, making r very large usually lowers BF_incl; if the result points the same way at r = 0.5 and r = 1, the evidence is robust to the prior.

What it does: The scale of the g ~ IG(1/2, r²/2) prior for fixed effects (JZS prior; Rouder et al. 2012; medium 0.5).

Effect on the result: The model BFs and the posterior change; the classical table does not.

On the validation data: PlantGrowth BF10: 3.439 for r = 0.25, 3.897 for 0.5, 3.659 for 0.707, 3.061 for 1. ToothGrowth full model: 2.906e14; 7.837e14; 9.729e14; 9.424e14.

→ JZS (Cauchy) prior and the scale parameter r

Covariates (Bayesian ANCOVA)

Concept: A covariate is a continuous variable whose influence is removed before the groups are compared. In Bayesian ANCOVA the covariate is also a separate term in the model space, and its own inclusion Bayes factor shows whether it is really needed.

Everyday example: If the plots differed in soil nitrogen before sowing, the fertiliser comparison is affected by that difference. Adding soil nitrogen as a covariate judges the fertilisers as if plots with equal nitrogen were being compared.

What it does: Continuous covariates enter the model space as separate terms (Zellner-Siow prior, rscaleCont √2/4); the model space is all subsets and inclusion BFs are given.

Effect on the result: The model BFs, inclusion BFs and the classical table (ANCOVA) change.

On the validation data: anorexia (covariate Prewt): model BFs Treat 61.81, Prewt 9.001, Treat+Prewt 364.1; inclusion BFs Treat 42.59, Prewt 5.941. The classical table is ANCOVA (Treat F = 7.868, BIC BF = 24.96).

→ Covariate

Subject column (repeated measures)

Concept: When the same subject is measured under several conditions, the measurements are dependent. The subject is added to every model as a 'nuisance' random effect, so overall level differences between subjects are not mixed up with the condition effect.

Everyday example: If all three fertilisers were tried on separate strips within each of the same 12 fields, every strip in a fertile field comes out high. Giving the field ID as the subject column lets the model judge fertiliser differences relative to each field's own level.

What it does: The subject is added to every model as a random "nuisance" effect (scale r = 1; Rouder et al. 2012); BFs are given only against the model with the subject.

Effect on the result: The BFs, the classical table (repeated-measures strata) and the posterior change.

On the validation data: puzzles: shape+ID 2.865, color+ID 2.865, shape+color+ID 11.89, full model 4.414. The classical table comes from the repeated-measures strata (shape F = 7.543, color F = 13.895).

→ Fixed and random effects

Additional output: Post-hoc, Assumption checks, Descriptives, Robustness

Concept: Posterior summaries show the size and uncertainty of the effects, assumption checks cover equal variances and normality, and the robustness table shows how much the results depend on the prior width. Bayesian post-hoc comparisons limit the risk of false alarms across many pairwise comparisons by adjusting the prior probabilities.

Everyday example: Once the fertiliser effect turns out strong, the question 'which fertilisers differ?' is answered with post-hoc comparisons: with BF = 15 between A and C and BF = 0.8 between A and B, A and C differ, while A and B cannot be told apart.

What it does: Posterior parameter summaries (deterministic with Gauss-Hermite), assumption checks (Levene in cells, Shapiro-Wilk on residuals), descriptives (cell n, mean, SD, 95% credible interval), robustness (effect BFs for r = 0.5, √2/2, 1) and, for a factor with 3+ levels, Bayesian post-hoc (JZS t, r = √2/2, prior odds correction of Westfall et al. 1997).

Effect on the result: Only adds output.

On the validation data: ToothGrowth Levene F = 1.940, p = 0.103; Shapiro-Wilk W = 0.985. Robustness for supp: 57.68; 58.29; 53.97. PlantGrowth trt1-trt2: corrected BF10,U = 6.390, posterior odds = 3.754 (prior odds 0.587 for J = 3).

→ Post hoc tests (pairwise comparisons)

Example on real data

Data: Crampton 1947 ToothGrowth, R datasets (dose labelled)

Variables: two-way mode, dependent len (tooth length), Factor 1 supp (VC / OJ), Factor 2 dose_lbl (three doses)

  • Question: How does tooth length (len) in guinea pigs change with vitamin C source (supp: OJ, VC), dose (0.5, 1, 2 mg) and their interaction? We grade the evidence for each effect with a Bayes factor.
  • Frequentist counterpart: supp F(1, 54) = 15.57, p < .001, η²p = .22; dose F(2, 54) = 92.00, p < .001, η²p = .77; interaction F(2, 54) = 4.11, p = .022, η²p = .13.
  • Result (JZS, r = 0.5): The most probable model is the full model (supp + dose + interaction, P(M|D) = .728, BF₁₀ = 7.84e+14); the posterior probability of the additive model is .267. The evidence is decisive for dose (BF_incl = 3.26e+14), very strong for supp (BF_incl = 143.3) and moderate for the interaction (matched BF = 2.73, BF_incl = 10.72).
  • BIC approach: supp BF₁₀ = 105.0, dose 1.40e+16, interaction 1.166; the BIC approach is more conservative than JZS for the interaction (weak evidence). A frequentist p = .022 and the strength of Bayesian evidence are not the same thing.
  • Descriptives / post-hoc: Length increases with dose (10.61, 19.74, 26.10); the uncorrected BF₁₀ is very large for all dose pairs (0.5-1: 81 800; 1-2: 953.6). In the interaction plot the advantage of OJ is clear at low doses and disappears at 2 mg.
  • APA: Bayesian two-way ANOVA (JZS, r = 0.5): full model BF₁₀ = 7.84e+14; supp BF_incl = 143.3, dose BF_incl = 3.26e+14, interaction BF_incl = 10.72; N = 60.

Related concepts

Post hoc tests (pairwise comparisons)JZS (Cauchy) prior and the scale parameter rFixed and random effectsPrior distributionPosterior distribution and posterior predictive checkCovariateBayes factor (BF₁₀)Interaction effect

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.