Statistics Glossary → Lack-of-fit test
Lack-of-fit testconcept
An F test of whether a model describes the data adequately compared with a full (saturated) model that gives every distinct combination of predictor values its own mean.
When there are repeated observations at the same predictor values, the residual sum of squares can be split into two parts. Pure error is the spread of observations around their own mean at each x value, which no model can explain. Lack of fit is the departure of those cell means from the model's predictions. The test statistic is F = MS_lack of fit / MS_pure error. A significant result indicates that the model's functional form (e.g. an assumed straight line or a model without interaction) is inadequate.
The test can only be computed when there are replicates; it is undefined when each x combination has a single observation, or for a saturated model that already gives each cell its own mean. In two-way ANOVA, the lack-of-fit test for a main-effects-only model corresponds to a test of the omitted interaction term. A non-significant result does not prove the model correct; the power of the test depends on the number of replicates.
An everyday example
A chemist sets the reaction temperature to 20, 40, 60 and 80 °C and measures the yield three times at each temperature. When she fits a straight line, the lack-of-fit test is significant: the three measurements at each temperature are very close together, but their means follow a curve that peaks at 60 °C rather than a straight line. The linear model cannot capture that curve.
In MerQur
In two-way ANOVA and ANCOVA, 'Model diagnostics: Lack of fit' tests the model against the full cell-means model with an F test; for a saturated model an 'undefined' note is given. Multiple linear regression also lists a lack-of-fit F test for replicated designs among its 'Model diagnostics'.
Analyses where this concept appears
Two-Way ANOVAANCOVAMultiple Linear Regression