Statistics Glossary → Crossed Mixed Model (Crossed LMM, Crossed Random Effects Mixed Model)
Crossed Mixed Model (Crossed LMM, Crossed Random Effects Mixed Model)analysis
A mixed model for designs in which two factors are crossed (every worker uses every machine, every judge scores every presentation), splitting the variability in the outcome into the two factors, their interaction and error, and assessing the reliability of the measurement.
Concept
In a crossed design every level of one factor appears together with every level of the other: each of six workers works on each of three machines, or every judge scores every pupil's presentation. Both factors then affect the outcome and may also interact; some workers may be especially good on a particular machine. A crossed mixed model measures these sources separately.
The model is y = A + B + A×B + e. One or both factors may be random: a factor is random if its levels are a sample from a wider population (randomly chosen workers) and fixed if they are all the levels of interest (the three machines to be compared). Expected mean squares (EMS) determine which term each effect is tested against; if both factors are random, for example, the main effects are tested against the interaction rather than against error. REML estimates the variance component of each source. A generalisability study (G-study) uses these components to compute coefficients showing how reliably the levels of one factor can be told apart when averaged over the levels of the other.
To separate the A×B interaction from error, each cell needs more than one observation; if each worker-machine combination is measured only once, the interaction is confounded with error. With few levels of a random factor (e.g. 3 varieties), its variance is estimated very imprecisely. With unbalanced data, EMS tests are approximate and the REML results should be relied on.
When to use it
- When two grouping factors are crossed and you want to separate how much each of them, their interaction and error contribute to the variability (worker × machine, rater × pupil, family × sex).
- To assess the reliability of a scoring or measurement process, a generalisability study is carried out with this model; if only agreement between raters is of interest, the intraclass correlation (ICC) can also be used.
- If the factors are nested rather than crossed, use a nested mixed model; if both factors are fixed, use classical two-way ANOVA.
An everyday example
A factory wants to know which of three machine types is most productive. Six randomly chosen workers each work three shifts on each machine, and hourly output is recorded. The crossed mixed model splits the differences in output into machine, worker, the worker × machine interaction and the error that varies from shift to shift. A large interaction means some workers do noticeably better on particular machines; the machine difference is then tested against the interaction, and the conclusion generalises beyond these six workers to the factory's workforce.
Parameters in MerQur
Variable selection: Dependent variable (DV), Factor A, Factor B
Concept: Selects the dependent variable and the two crossed factors. If each cell has on average at least two observations, the A×B interaction is estimated as a separate term; with a single observation, the interaction cannot be separated from error.
Everyday example: In the factory example the dependent variable is hourly output and the factors are machine and worker. Because each worker does three shifts on each machine, the worker × machine interaction can be separated from shift-to-shift error; with a single shift it could not.
What it does: Builds the model y = A + B + A×B + e. If the mean number of observations per cell is 2 or more, A×B is a separate term; otherwise it is confounded with the error and a warning is given (Penicillin).
Effect on the result: Determines all tables.
On the validation data: Machines Type I SS: machine 1755.263, worker 1241.895, A×B 426.530, error 33.287 (36 df). In the unbalanced crossed trial data (52.8) the Type I and Type III SS are given separately.
Factor A type / Factor B type (Random (random sample) / Fixed (constant / intervention))
Concept: Determines whether each factor is treated as random or fixed. A random factor's levels are a sample from a population and the conclusions generalise to it; a fixed factor's levels are all the levels of interest. The choice changes the denominators of the F tests and the variance components.
Everyday example: If the three machine types are all the machines the factory is considering buying, machine is fixed. If the six workers were drawn at random from all employees, worker is random, so the conclusions apply 'to the factory's workers', not just 'to these six people'.
What it does: Sets whether the factor levels are a random sample from a population or all levels of interest; determines the EMS error terms and the factor's role in the REML model. Both cannot be fixed (classical Two-Way ANOVA is recommended).
Effect on the result: Changes the F denominators, REML components and G-study coefficients.
On the validation data: Machines, both random: machine F = 20.576, worker F = 5.823 (against MS_AB); REML σ² machine 46.39, worker 22.86, A×B 13.91, e 0.925; Eρ² = 0.951, Φ = 0.882.
SS method (Type I (sequential) / Type III (partial - for unbalanced designs))
Concept: Determines how the sum of squares is partitioned. Type I adds terms sequentially; Type III measures each term's contribution with all other terms in the model. With balanced data they agree.
Everyday example: If every worker did exactly three shifts on every machine, the two methods agree. If some shifts went unrecorded, the results differ and Type III is preferred.
What it does: Sequential (Type I) or partial (Type III) sums of squares.
Effect on the result: Changes the result only for unbalanced data.
On the validation data: In balanced Machines Type I and Type III are identical. In the unbalanced 52.8 data machine F: Type I 20.388, Type III 15.312; in heights Family F 1.643 -> 2.045.
EMS model (mixed design) (Unrestricted (default; SPSS/SAS) / Restricted (Cornfield-Tukey); only when one factor is fixed and one random)
Concept: Two theoretical approaches to testing the random main effect when one factor is fixed and the other random. In the unrestricted model the random effect is tested against the interaction, in the restricted model against error; the fixed factor is tested against the interaction in both.
Everyday example: With machine fixed and worker random, the test of 'do workers differ?' can give a much larger F under the restricted model. The approach used should be stated in the report; many common software packages use the unrestricted model.
What it does: In the unrestricted model the EMS of the random main effect contains σ²_AB and the denominator is MS_AB; in the restricted model it does not and the denominator is MS_e. The fixed factor is tested against MS_AB in both.
Effect on the result: Changes the F of the random main effect. The field is disabled when both factors are random.
On the validation data: Worker F: 5.823 (unrestricted) -> 268.625 (restricted). Heights Type III restricted: Family F = 5.901, A×B F = 2.885.
Covariates (REML model)
Concept: Adds a numeric fixed effect to the REML model that estimates the variance components, removing the effect of a variable that influences the outcome but is not one of the factors.
Everyday example: If the temperature during a shift affects output, temperature is added as a covariate. The part explained by temperature is removed from the residual variance, and the worker and machine components become clearer.
What it does: Added only to the REML crossed model as numeric fixed effects; the EMS table stays without covariates.
Effect on the result: Changes the REML components and fixed effects; does not change the EMS table or the G-study coefficients.
On the validation data: Adding nitrogen in Oats: σ²(Variety x Block) 19.26 -> 108.94, σ²(e) 524.28 -> 165.56; nitrogen coefficient 73.667 (SE 6.781, df 53). Eρ² = 0.327 stays the same.
Fixed-factor comparisons ((none) / Tukey HSD / Bonferroni / Šidák / Unadjusted (LSD))
Concept: Compares the level means of the fixed factor in pairs, using the error term from that factor's F test; a multiple-comparison correction limits the risk of false alarms.
Everyday example: For three machines there are three pairwise comparisons. Because worker is random, these comparisons rest on the worker × machine interaction; if machine differences vary a lot between workers, only large differences are significant.
What it does: Tests pairwise differences of fixed-factor level means with the error term of that factor's F test (MS_AB, df_AB if A is fixed and B random); produces letter groups.
Effect on the result: Changes only the comparison table. Skipped with a warning when both factors are random. For unbalanced data raw means are used with a warning.
On the validation data: Machines A - B difference -7.967, SE 2.177, t(10) = -3.660, p = 0.00439; Tukey 0.01114, Bonferroni 0.01318, Šidák 0.01312.
Confidence Level (90% / 95% / 99% / Other (custom))
Concept: Sets the confidence level of the intervals for the level means and variance components; the higher the level, the wider the interval.
Everyday example: Machine C's mean hourly output might lie between 64 and 68 items with 95% confidence; at 99% the interval widens.
What it does: Sets the t interval of the A and B level means and the log-Wald interval of the REML variance components.
Effect on the result: Only the interval limits change.
On the validation data: Machine C mean 66.272: 95% [64.186, 68.358]; 90% [64.552, 67.992]; 99% [63.407, 69.138].
REML section (in every run)
Concept: Gives the REML variance component of each source of variability (A, B, A×B, error) and its uncertainty. The components show how much of the total variability comes from each source and are used to compute the generalisability coefficients.
Everyday example: The components might show, for example, that 55% of the differences in output come from the machine, 25% from the worker, 15% from the worker-machine fit and 5% from shift-to-shift variability.
What it does: Gives σ² components, standard errors, Wald Z, log-Wald intervals, fixed effects (Satterthwaite df) and AIC/BIC.
Effect on the result: Given in every run next to the EMS table; independent of the options.
On the validation data: Penicillin plate 0.71691, sample 3.73097, e 0.30241 (REML criterion 330.9); heights Family 2.4010, F x G 1.7657, e 2.1668 (-2RLL 71.0).
EMMEANS factor ((none) / Factor A / Factor B) and EMMEANS p adjustment (Bonferroni / Šidák / Tukey / No adjustment)
Concept: Estimated marginal means are model-based level means with the other effects balanced out; they are given only for a fixed factor. Pairwise comparisons are made with a multiple-comparison correction.
Everyday example: If some workers' shifts on some machines are missing, the raw machine means may be skewed by which workers spent more time on which machine. The estimated marginal mean corrects this imbalance.
What it does: If the selected factor is fixed, computes its estimated marginal means and pairwise comparisons from the REML crossed model with Satterthwaite df. No EMM is given for a random factor.
Effect on the result: Only EMM and pairwise tables are added; the EMS table, F tests, REML components and G-study do not change. The adjustment changes only the pairwise p values.
On the validation data: In independent crossed trial data (A fixed, B random) EMM, SE, df and Bonferroni-adjusted p values are given.
Example on real data
Data: Yates 1935 oats trial, nlme Oats
Variables: dependent yield, factor A Variety, factor B Block, both random
- Question: How much does oat yield vary between randomly sampled varieties (Variety, 3) and blocks (Block, 6)? The two factors are crossed and both are treated as random.
- Assumption: The random variety variance is estimated with 2 degrees of freedom and is therefore very uncertain (SE = 38.87); because the nitrogen dose in the data (nitro) is not in this model, the residual variance is large.
- Result: Under EMS the main effects are tested against the interaction mean square: the variety effect is not significant (F(2, 10) = 1.49, p = .272), the block effect is significant (F(5, 10) = 5.28, p = .012); the variety x block interaction is not significant (F(10, 54) = 1.15, p = .346). Variety means: Marvellous 109.79, Golden Rain 104.50, Victory 97.63.
- Effect size: σ² variety = 12.16 (ICC .016), σ² block = 214.48 (.278), σ² variety x block = 19.26 (.025), σ² error = 524.28 (.681). In the G-study the relative Eρ² = 0.327 and absolute Φ = 0.167; with 6 blocks the varieties cannot be reliably distinguished.
- APA: Block F(5, 10) = 5.28, p = .012; variety F(2, 10) = 1.49, p = .272; σ²_block = 214.48, σ²_error = 524.28.
Related concepts
F testExpected mean squares (EMS)Restricted maximum likelihood (REML)Variance componentsFixed and random effectsEstimated marginal means (EMM)Nested and crossed random effectsTypes of sums of squares (Type I, II, III)
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.