Statistics Glossary → MANOVA
MANOVAanalysis
A multivariate analysis of variance that tests in a single test whether groups differ on several continuous outcomes taken together, that is, on the profile formed by the means of those outcomes.
Concept
An intervention or a group difference often shows up not in one outcome but in several related outcomes at once. Running a separate ANOVA for each outcome both inflates the false-alarm risk and ignores the relationships between the outcomes. MANOVA treats the dependent variables as a whole and tests the null hypothesis that all groups have equal mean vectors (the profiles formed by the means on each variable). Sometimes differences that are not clear on any single variable emerge when the variables are considered together.
The between-group and within-group sums of squares of ANOVA are replaced here by matrices: the between-group (H) and error (E) sums-of-squares-and-cross-products matrices. Four test statistics are derived from the ratio of these two matrices: Wilks' Λ (the ratio of within-group to total variability; a small value means a large group difference), Pillai's trace, the Hotelling-Lawley trace and Roy's largest root. Each is converted to an F distribution to obtain a p value. Pillai's trace is the most robust to violations of the assumptions.
Assumptions: observations are independent, the dependent variables are jointly multivariate normal within each group, the groups' covariance matrices are equal (Box's M), and there are no multivariate outliers. MANOVA makes most sense when the dependent variables are moderately correlated; if they are almost uncorrelated, separate ANOVAs, and if they are very highly correlated, a single composite score, may be more suitable. After a significant MANOVA, follow-up ANOVAs, post-hoc tests or discriminant analysis show on which variables and between which groups the difference lies.
When to use it
- When groups (e.g. three training programmes, three treatments) are to be compared on several related continuous outcomes together.
- With only one outcome variable, one-way or two-way ANOVA is enough; if the same outcome is measured several times on the same people, use repeated measures ANOVA.
- If the combination of variables that best separates the groups is itself of interest, use discriminant analysis; if multivariate normality and equal covariances are badly violated, consider permutation-based multivariate tests (e.g. PERMANOVA).
An everyday example
A sports centre wants to compare three programmes (yoga, Pilates, aerobics) and measures the flexibility, muscle strength and endurance scores of 25 members from each programme after 12 weeks. Rather than testing the three scores separately, MANOVA tests in one go whether the programmes differ in the 'fitness profile' formed by these three scores. If Wilks' Λ = 0.70 with p < 0.001, the programmes' profiles differ; follow-up ANOVAs may show that the difference lies mainly in flexibility (yoga ahead) and endurance (aerobics ahead).
Parameters in MerQur
Confidence Level (%90, %95, %99, Other/custom 80.0-99.9)
Concept: The confidence level sets the threshold for significance decisions (α = 1 − confidence level). It does not change the test statistics or p values; it only changes which results count as 'significant'.
Everyday example: If the programme difference gives p = 0.02, it is significant at the 95% level and not significant at the 99% level. If the sports centre wants its investment decision to rest on firm evidence, it may choose a higher confidence level.
What it does: α = 1 - confidence level; MANOVA produces no confidence interval.
Effect on the result: Only the decision statements (H0 reject / fail to reject, Box M and Levene "met / VIOLATED") and the significance flags of the post-hoc and univariate rows change; the statistics do not. The custom value is read only when "Other" is selected.
On the validation data: hsb2 Box M p = 0.1663: "met" at 95%, "VIOLATED" at 80% (α = 0.20). UCLA Pillai p = 0.0122: "H0 cannot be rejected" at 99%.
Reference test (Wilks, Pillai, Hotelling-Lawley, Roy)
Concept: The four multivariate tests summarise the same question in different ways. Wilks' Λ is the most commonly reported; Pillai's trace is the most robust to assumption violations and unbalanced groups; the Hotelling-Lawley trace usually gives results similar to Wilks'; Roy's largest root is the most powerful when the group difference is concentrated in one dimension but the most prone to false alarms when assumptions fail. The tests usually lead to the same decision.
Everyday example: If the difference between programmes lies essentially along a single 'flexibility axis', Roy's test picks it up most easily. If group sizes are unbalanced and Box's M is borderline, the decision is based on Pillai's trace.
What it does: Selects which multivariate test gives the overall decision (the F, p and decision in the heading); the table always shows all four tests.
Effect on the result: The heading F, p and decision change.
On the validation data: UCLA: Wilks 0.52578838, F(6, 56) = 3.5382, p = 0.004859; Pillai 0.47667013, F(6, 58) = 3.0248; Roy 0.89198790, F(3, 29) = 8.6225. Pillai is recommended for unbalanced or small groups or when covariance homogeneity is in doubt (Olson, 1979).
→ Multivariate test statistics (Wilks' Λ, Pillai's trace, Hotelling-Lawley, Roy)
Effect size, univariate ANOVA, descriptive (checkboxes)
Concept: The multivariate effect size shows how much of the variability in the combination of dependent variables is explained by the group difference. The follow-up univariate ANOVAs look at each variable separately; because several tests are made, their p values are adjusted by multiplying by the number of variables (Bonferroni). The descriptive table gives each group's mean on each variable.
Everyday example: An effect size of 0.16 means that about 16% of the variation in the fitness profile is related to the programme. If separate ANOVAs for the three scores give p = 0.01 for flexibility, Bonferroni turns this into 0.03, which is still significant at the 5% level.
What it does: Effect size 1 - Λ^(1/s); for each dependent variable F, p, Bonferroni p (x number of variables) and partial η²; group means.
Effect on the result: Extra tables only.
On the validation data: hsb2 effect size 0.1433; read F = 21.282 (η²ₚ = 0.1777), write F = 21.275 (η²ₚ = 0.1776), math F = 29.279 (η²ₚ = 0.2291). UCLA useful F(2, 30) = 2.701, p = 0.0835, Bonferroni 0.2504.
Box M (checkbox)
Concept: Box's M test checks whether the groups' covariance matrices, that is, the variance of each variable and the relationships between the variables, are the same across groups. It is very sensitive and finds trivial differences significant in large samples, so it is usually judged against a strict threshold such as 0.001.
Everyday example: If flexibility and muscle strength are strongly related in the yoga group but only weakly in the aerobics group, the groups' covariance structures differ. If Box's M is significant and group sizes are also unbalanced, it is safer to rely on Pillai's trace.
What it does: Tests the equality of the group covariance matrices with the χ² approximation.
Effect on the result: Information row only. When a group has n <= p, "could not be computed" is written.
On the validation data: hsb2 χ² = 16.574, df = 12, p = 0.1663; UCLA χ² = 10.286, p = 0.5909.
Follow-up ANOVA post hoc (None, Tukey, Bonferroni, Games-Howell)
Concept: When a follow-up ANOVA finds a group difference on a variable, post-hoc tests show which pairs of groups differ on that variable while controlling the multiple-comparison risk. Tukey suits equal variances and Games-Howell unequal ones; Bonferroni is a more conservative option.
Everyday example: If the programme difference in flexibility is significant, Tukey's test might conclude that 'yoga is higher than both Pilates and aerobics, with no difference between Pilates and aerobics'.
What it does: Gives the difference and p of the group pairs for each dependent variable.
Effect on the result: Effective only when the univariate ANOVA is on.
On the validation data: UCLA useful: Tukey control_2 - treatment p = 0.1117; Bonferroni 0.1393; Games-Howell 0.1848.
Second factor and covariates
Concept: When a second factor is added, factorial MANOVA tests the main effects of the two factors and their interaction multivariately. When a numeric covariate is added (MANCOVA), group comparisons are adjusted for that covariate.
Everyday example: If gender is added as a second factor, it tests whether the programmes' effects differ between women and men. If age is added as a covariate, the programmes are compared 'for members of the same age', so a younger aerobics group does not distort the result.
What it does: Computes a g x f2 factorial MANOVA and/or MANCOVA (Type III, sum-to-zero contrast) as a separate block. If the group factor is chosen as the second factor, no block is produced.
Effect on the result: The main one-way table does not change.
On the validation data: tear: rate Wilks 0.3819, F(3, 14) = 7.5543; additive 0.5230, F = 4.2556; interaction 0.7771, F = 1.3385. hsb2 + science: prog Wilks 0.7515, F(6, 388) = 9.9310.
Additional diagnostics: partial η² (each test), Levene (per variable), Box M F + Bartlett + SSCP, Mardia + Mahalanobis (checkboxes)
Concept: The additional diagnostics show the assumptions and the structure of the model in detail. Partial η² gives the effect size of each multivariate test; Levene's test checks equal variances on each variable; Bartlett's sphericity test checks whether the dependent variables are correlated enough to justify MANOVA; the H and E matrices show between-group and within-group variability; Mardia's measures assess multivariate normality, and Mahalanobis distances point to multivariate outliers that are unusual in their combination of values.
Everyday example: If a member has a very high flexibility score but very low muscle strength, each score on its own may lie in the usual range, yet the combination is unusual; the Mahalanobis distance flags this member. If Bartlett's test is not significant, the three scores are nearly uncorrelated and separate ANOVAs may be enough.
What it does: Partial η² for each test; Levene for each dependent variable; the Box M F approximation, the Bartlett sphericity test of the residual correlation matrix, the H and E SSCP matrices; Mardia skewness/kurtosis and Mahalanobis distances on the residuals.
Effect on the result: Extra sections only. When a group has n <= p, the Box M F approximation is reported as not computed.
On the validation data: UCLA partial η² Wilks 0.2749, Roy 0.4715; Levene useful F(2, 30) = 4.8783, p = 0.0146; Box M F(12, 4361.54) = 0.8544. iris Mardia skewness χ²(20) = 31.8481, p = 0.044944.
Example on real data
Data: UCLA OARC hsb2 (High School and Beyond, n = 200)
Variables: dependents read, write, math (reading, writing, mathematics scores), group prog (programme type)
- Question: Does the composite profile of reading, writing and mathematics scores differ by programme type (prog: general, academic, vocational)?
- Assumption: Box M χ²(12) = 16.57, p = .166; homogeneity of the covariance matrices is met.
- Result: Wilks Λ = 0.734, F(6, 390) = 10.87, p < .001. The null hypothesis is rejected: the programme types differ on the combination of the three scores.
- Effect size: Multivariate partial η² (1 - Λ^(1/s)) = 0.143; a medium-to-large effect.
- Follow-up ANOVAs: Each of the three dependent variables is significant on its own (p < .001 also with Bonferroni ×3): read F(2, 197) = 21.282 (η²p = .1777), write F(2, 197) = 21.275 (η²p = .1776), math F(2, 197) = 29.279 (η²p = .2291); the read and write values are very close to each other; the largest separation is in mathematics.
- APA: Wilks' Λ = 0.734, F(6, 390) = 10.87, p < .001, η²p = .14.
Related concepts
Effect sizeLevene's testPost hoc tests (pairwise comparisons)Box's M test (equality of covariance matrices)Multivariate test statistics (Wilks' Λ, Pillai's trace, Hotelling-Lawley, Roy)Between-subjects factorEta squared (η²) and partial eta squared (η²ₚ)Mahalanobis distance
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.