MerQuR+Statistics Glossary

Statistics Glossary → Variance Components Analysis (VARCOMP)

Variance Components Analysis (VARCOMP)analysis

An analysis that splits the total variability in a measurement into sources treated as random effects (such as batches, operators, schools, animals) and measurement error, estimating the share of each source.

Concept

The quality of a product, a pupil's score or an animal's yield is affected by many sources at different levels. Rather than comparing means, variance components analysis asks: how much of the variability comes from differences between batches, how much from differences between units within a batch, and how much from measurement error? This information is needed to decide where to direct improvement efforts (quality control), to assess the adequacy of a measurement system (GRR), or to estimate the heritability of a trait.

The model writes each observation as the overall mean plus a random deviation from each source: y = μ + a(batch) + b(unit within batch) + e. Each deviation has its own variance, and the total variance is their sum: σ²(total) = σ²(a) + σ²(b) + σ²(e). The factors are treated as random because the interest is not in the particular batches in the sample but in the variability in the population of batches they come from. If a factor's levels make sense only within a higher factor (each batch has its own vats), the factors are nested; if every level can combine with every other (each operator measures each part), they are crossed.

The components are estimated by REML, ML or moment methods based on expected mean squares; REML is generally recommended. Moment methods can give negative variance estimates, a sign that the component is in fact very small. For factors with few levels (e.g. only 5-10 batches) that factor's variance is estimated very imprecisely; to test whether a component is zero, the likelihood ratio test is more trustworthy than the Wald test. Results are usually reported as each component's percentage of the total variance.

When to use it

  • When the aim is not to compare means but to find the sources of variability and their shares: batch and machine variability in production, measurement system analysis, inter-laboratory reproducibility, heritability studies.
  • If the specific levels of a factor are themselves of interest (e.g. comparing just these three machines), use fixed-effects ANOVA.
  • If testing fixed effects and making predictions alongside the random effects is central, use linear mixed models; if only agreement between raters is asked, the intraclass correlation coefficient is a more direct tool.

An everyday example

At a dairy, the thickness of the yogurt varies from day to day. The quality manager selects 10 production batches, takes three vats from each batch and makes two measurements from each vat. Variance components analysis might show that, say, 15% of the total variability comes from differences between batches (milk supplier, season), 78% from differences between vats within the same batch, and only 7% from measurement error. This tells the manager to look for the problem not in the milk supply but in the vats' heating or stirring settings.

Parameters in MerQur

Dependent (numeric DV) and Random factors (first factor)

Concept: The dependent variable is the numeric measurement whose variability is studied; a random factor is a grouping variable whose levels are regarded as sampled from a larger population and whose own variance is to be estimated. For a random factor, the interest lies not in the means of individual levels but in how much the levels vary.

Everyday example: Yogurt thickness is the dependent variable and production batch the random factor. The quality manager asks not 'why was batch three bad?' but 'how much does thickness vary from batch to batch?'.

What it does: Selects the dependent variable and at least one random factor.

Effect on the result: All components.

On the validation data: Dyestuff (Batch): REML component 1764.05; ML 1388.33. The Hemmerle-Hartley REML values are 1464.368 / 26.959 / 78.842.

→ Fixed and random effects

Random factors: second factor and nested in upper factor

Concept: A second random factor can be nested within or crossed with the first. In a nested structure the levels of the lower factor exist only within a particular level of the upper factor; in a crossed structure every level combines with every other. Choosing the wrong structure splits the components incorrectly.

Everyday example: If each batch has its own vats and 'vat 1' refers to a different vat in each batch, vat is nested within batch. If the same three lab technicians measure samples from every batch, technician and batch are crossed.

What it does: Adds a second random effect. If the box is ticked, the levels of the second factor are meaningful only within each level of the upper factor (group identifier f1::f2); if not, the two factors are treated as crossed. If the lower-unit codes repeat within the upper unit, nested must be selected (Pinheiro and Bates, 2000).

Effect on the result: The number of variance components and the partition.

On the validation data: Pastes: nested REML batch 1.6573, cask(batch) 8.4336, residual 0.678. With the box off (crossed), batch 3.3639, cask 0.1487, residual 7.306: the wrong model for these data.

→ Nested and crossed random effects

Estimation method (REML (recommended), ML, ANOVA (Type I, EMS), MINQUE(0) / MIVQUE0, MINQUE(1))

Concept: Variance components can be estimated on different principles. REML corrects the bias that comes from estimating the fixed effects and is generally recommended; ML is used when comparing models with different fixed effects by likelihood; moment methods such as ANOVA and MINQUE assume no distribution but can give negative estimates.

Everyday example: With only 10 batches, ML systematically underestimates the batch variance a little; REML corrects this. If the ANOVA method gives something like −0.3 for the batch variance, read it as the difference between batches being practically zero.

What it does: Selects one of REML (default), ML, ANOVA Type I (expected mean squares equations), MINQUE(0) = MIVQUE0 and MINQUE(1). REML is the general recommendation (Searle et al., 1992); ML serves to compare models with different fixed effects; the moment methods assume no distribution but can give a negative component.

Effect on the result: Component values; in REML and ML also the SE, Wald Z, log-Wald confidence interval, asymptotic covariance and information criteria; in ANOVA the expected mean squares table.

On the validation data: Dyestuff (balanced) REML = ANOVA = MINQUE0 = MINQUE1 = 1764.05, ML 1388.33. Dyestuff2: REML and ML Batch 0 (boundary), ANOVA and MINQUE -1.3219 (negative). Hemmerle-Hartley (unbalanced) Var(b): REML 1464.37, ML 723.67, Type I 1448.38, MIVQUE0 1466.12, MINQUE1 1473.61.

→ Restricted maximum likelihood (REML)

Fixed factors / covariates

Concept: Fixed factors and covariates are known systematic effects that should be kept separate from the random components. Once they are added to the model, the random components show the variability left after removing these effects.

Everyday example: If some batches were produced in summer and some in winter, and season is known to affect thickness, season is added as a fixed factor. The batch variance then reflects differences between batches in the same season, not the seasonal difference.

What it does: Adjusts the components for the fixed effects. A text column enters as a treatment-coded factor (first level is the base level), a numeric column as a covariate; a fixed-effects table with Satterthwaite degrees of freedom is given. The same column as a random factor cannot be selected as fixed (an explicit error is raised).

Effect on the result: Components.

On the validation data: Hemmerle-Hartley, a_c (3 levels) fixed: REML Var(b) 1464.37, Var(a(b)) 26.96, residual 78.84. If the digit-text a_f is converted to a number (1 df covariate), 1430.75 / 163.80 / 78.27 results.

→ Covariate

Categorical (factor) variables

Concept: A variable entered as numeric codes should be treated as categories rather than a continuous measurement if the codes merely label groups (1 = supplier A, 2 = supplier B). Otherwise the model assumes a meaningless linear relation across the codes.

Everyday example: If milk suppliers are coded 1, 2 and 3, assuming supplier 3 has an effect 'two units larger' than supplier 1 makes no sense. Declaring supplier as categorical gives each supplier its own effect.

What it does: Columns written separated by commas enter as levels (with dummy coding) even if they are numerically coded.

Effect on the result: These columns are treated as factors instead of covariates; if the same column remains in the fixed list, it is treated as a covariate and a note is added.

On the validation data: Hemmerle-Hartley, a_f as a factor: REML Var(b) 1464.368, Var(a(b)) 26.959, residual 78.842; the asymptotic covariance matrix is also given.

→ Dummy (indicator) variable coding

Add the interaction component of the first two crossed random factors (A×B)

Concept: The interaction component captures extra variability specific to combinations of two crossed random factors, i.e. one factor's effect changing with the level of the other. To separate this component from the residual, each cell needs more than one measurement.

Everyday example: If technicians measure some batches systematically differently, for instance one technician reading high only on thick yogurts, that is a technician × batch interaction. In measurement system analysis this component is an important part of measurement reproducibility.

What it does: Adds the interaction component of the first two crossed random factors; applies to measurement system (GRR) and crossed designs with replication per cell.

Effect on the result: The number of components and the partition of the residual. With a single observation per cell the interaction cannot be separated from the residual and an explicit error is raised.

On the validation data: Houf (3 replicates in each cell): a 48.2926, b 0.5646, a x b 0.7280, residual 0.5111; with the option off a 48.4792, b 0.6206, residual 1.0151 (the interaction is absorbed into the residual).

→ Interaction effect

Likelihood-ratio test of σ² = 0 for each component (REML/ML)

Concept: The likelihood ratio test checks whether a component is zero by seeing whether fit worsens significantly when that component is removed. Because a variance cannot be negative, zero lies on the boundary of possible values; the ordinary chi-square p value is therefore too conservative, and a p value based on the boundary mixture is preferred.

Everyday example: If removing the batch component leaves the fit almost unchanged, real variability between batches cannot be demonstrated. With only 10 batches, this test gives a more trustworthy answer than the Wald test.

What it does: Drops each component and refits with the same method; gives the chi-square(1) p value and the boundary mixture (half chi-square(1)) p value. Because chi-square is conservative at the boundary, the mixture p value is recommended (Self and Liang, 1987). Disabled when ANOVA and MINQUE are selected.

Effect on the result: The test table only.

On the validation data: Dyestuff REML: LRT 6.369, p = 0.0116, mixture p = 0.0058. Pastes ML batch: LRT 0.4072, p = 0.5234.

→ Likelihood ratio test (LRT)

Example on real data

Data: Davies and Goldsmith 1972 Pastes, lme4 package

Variables: dependent strength (paste strength), random factors batch and cask nested within batch

  • Question: How is the variability in paste strength (strength) shared among batches (batch), casks within batches (cask) and measurement error?
  • Model: Nested random effects, REML; N = 60 (10 batches x 3 casks x 2 measurements); -2LL (REML) = 246.99, AIC = 254.99, BIC = 263.37.
  • Result: σ²(cask within batch) = 8.434 (78.3%), σ²(batch) = 1.657 (15.4%), σ²(residual) = 0.678 (6.3%); total variance 10.77.
  • Inference: The cask component is significant (Wald Z = 3.04, p = .002, 95% CI [4.42, 16.07]); the batch component is not (Wald Z = 0.71, p = .481, CI [0.10, 26.67]).
  • Interpretation: Most of the variability lies between casks (ICC = .78); the difference between batches is smaller. In quality control, the cask level is the source to focus on.
  • Limitation: Wald Z is not reliable for components with few levels (10 batches); the likelihood-ratio test should be preferred.
  • APA: With REML, 78.3% of the variance is in the cask (within batch), 15.4% in the batch and 6.3% in the residual component; σ² cask = 8.43, SE = 2.78, z = 3.04, p = .002.

Related concepts

Likelihood ratio test (LRT)Restricted maximum likelihood (REML)Fixed and random effectsNested and crossed random effectsCovariateInteraction effectHeritability (h²)Types of sums of squares (Type I, II, III)

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.