Statistics Glossary → Normality Tests
Normality Testsanalysis
Tests and plots that check whether the values of a numeric variable could plausibly have come from a normal (bell-shaped) distribution.
Concept
Many parametric methods, such as the t-test, ANOVA and linear regression, assume that the data (or the model residuals) are approximately normal. Normality tests check how reasonable this assumption is and help decide whether to proceed with a parametric method or with its nonparametric counterpart.
In every normality test the null hypothesis is that the data come from a normal distribution. The tests look from different angles: Shapiro-Wilk measures how well the sorted data match the expected ordered values of a normal distribution; Kolmogorov-Smirnov and Anderson-Darling compare the observed cumulative distribution with the normal cumulative curve (Anderson-Darling giving more weight to the tails); D'Agostino's K² and Jarque-Bera look at how far skewness and kurtosis depart from their normal values. If p < α, normality is rejected.
The power of these tests depends heavily on sample size. In a small sample even clear departures may go undetected, while in a large sample practically trivial departures become significant. Failing to reject normality does not prove it. The decision should therefore rest not on the p value alone but also on the Q-Q plot, the size of the skewness and kurtosis, and how robust the intended method is to non-normality.
When to use it
- Before applying a method that assumes normality, such as a t-test, ANOVA or regression, to check the distribution of the variable or of each group.
- In small and medium samples Shapiro-Wilk is usually the most powerful choice; if departures in the tails matter, Anderson-Darling may be preferred. In very large samples, look at the Q-Q plot and the size of the departure rather than the test result.
- If normality is clearly violated and the sample is small, consider nonparametric methods such as the Mann-Whitney U, Wilcoxon or Kruskal-Wallis tests, or a suitable transformation of the data.
An everyday example
A bakery weighs 40 randomly chosen loaves in one day to monitor bread sold as 500 grams. Before comparing the mean weight with the target using a t-test, it checks whether the weights are spread in a roughly bell-shaped way. If the Shapiro-Wilk test gives p = 0.41, there is no clear departure from normality and the t-test can go ahead. But if one oven keeps underbaking its loaves, leaving them noticeably heavier, the distribution stretches to the right, the test gives p = 0.003, and on the Q-Q plot the points at the right end break away from the line.
Parameters in MerQur
Column (list of numeric columns)
Concept: Normality is a property of a variable's distribution, so the test is run on one numeric variable at a time. Missing values carry no information about the distribution and are left out.
Everyday example: The bakery's data file holds loaf weight, baking time and oven number. If the distribution of weight is the question, the test is applied to the weight column; two loaves that were never weighed drop out of the calculation.
What it does: Selects the numeric column to be tested; missing values are dropped (airquality Ozone n = 116).
Effect on the result: All tests are applied to the selected column.
On the validation data: In 22 columns, n, mean, SD, skewness (G1), kurtosis (G2) and z values are computed; for enroll G1 1.349 (SE .122), G2 3.108 (SE .243).
α (alpha)
Concept: The significance level α is the false-alarm risk accepted for a 'not normal' decision. If the p value is below α, normality is rejected; lowering α makes the decision more cautious but raises the risk of missing real departures.
Everyday example: With α = 0.05, loaf weights giving p = 0.03 count as 'not normal'; with α = 0.01 the same result would be accepted as 'normal'. That is why α must be fixed before looking at the result.
What it does: In each test row p > α gives "Normal", otherwise "Not normal"; the overall decision is summarized as all normal, none normal or mixed (Uncertain).
Effect on the result: The statistic and p do not change; only the flags and the overall decision change.
On the validation data: For hsb2 math, S-W p .0021, K-S p .0160, D'Agostino p .0067: "Not normal" at α .05 and .10, Uncertain at α .01 (Lilliefors normal, the others not).
By group ((No grouping) / categorical or numerically coded factor)
Concept: Methods such as the t-test and ANOVA assume normality within each group, so normality is tested separately for each group rather than on the pooled data. A mixture of two groups can look non-normal even when each group is normal on its own.
Everyday example: If the bakery has two ovens, one producing loaves averaging 495 g and the other 510 g, all loaves together may form a two-peaked distribution. Tested separately, each oven's loaves may well be normal.
What it does: Runs the same tests separately at each group level.
Effect on the result: The table tab is in group x test form.
On the validation data: hsb2 write x female: female n = 109, S-W W .9459, p .0002; male n = 91, W .9499, p .0015. PlantGrowth 3 x 10: S-W p .7475 / .4519 / .5643.
Tests: Shapiro-Wilk, K-S (Lilliefors), Anderson-Darling, D'Agostino K², Jarque-Bera, Cramér-von Mises, Shapiro-Francia (checkboxes)
Concept: Each test is sensitive to a different aspect of non-normality: Shapiro-Wilk and Shapiro-Francia to how well the ordered values fit, Kolmogorov-Smirnov, Anderson-Darling and Cramér-von Mises to differences between cumulative distributions, and D'Agostino's K² and Jarque-Bera to skewness and kurtosis. When the tests disagree, the departure is borderline or of a particular kind.
Everyday example: If only a few loaves are very heavy, the tail-sensitive Anderson-Darling test may pick this up while Kolmogorov-Smirnov misses it. For a sample of 40 loaves, Shapiro-Wilk on its own is usually a good start.
What it does: If none is checked, selection is automatic by n (n < 50 S-W; 50-299 S-W + K-S; n >= 300 K-S + A-D; D'Agostino additionally if n >= 20). If checked, only the selected tests run; tests below their minimum n and S-W/S-F at n > 5000 are skipped with the reason stated.
Effect on the result: Only which tests are reported and the overall decision change; each test's statistic and p value are independent of the other selections.
On the validation data: hsb2 write (n = 200): S-W W .94703, p 9.87e-07; Lilliefors D .13433, p 2.21e-09; Anderson-Darling A² 3.25904, p 3.444e-08; D'Agostino K² 18.9923, p 7.51e-05; Jarque-Bera 12.4614, p .00197.
Second chart (Q-Q, Detrended Q-Q, P-P, Box plot)
Concept: Normal probability plots make visible the information a test reduces to a single number. On a Q-Q plot normal data line up along a straight line; the detrended Q-Q plot magnifies departures from that line around a horizontal axis; the P-P plot compares cumulative probabilities and is more sensitive to departures in the middle of the distribution; the box plot shows skewness and outliers.
Everyday example: If the points on the Q-Q plot of loaf weights follow the line in the middle but curl upwards at the right end, the problem is a few heavy loaves. On the box plot those loaves appear as separate points beyond the upper whisker.
What it does: Draws one of the normal probability plot types. Q-Q, detrended Q-Q and P-P use Blom positions and the standardized observation; the normal curve on the histogram uses σ = s (ddof = 1).
Effect on the result: Chart only; the tests do not change.
On the validation data: For hsb2 write the normal curve has σ = 9.4786.
Batch test (extra columns)
Concept: Applying the same normality check to several variables at once. When many tests are run, keep in mind that some may come out significant by chance alone.
Everyday example: If the bakery measured loaf length and crust thickness as well as weight, the normality of all three is checked in one go. If ten variables are tested, then even if all are normal, at α = 0.05 about half of one is expected to come out 'not normal' by chance.
What it does: Produces the same tests for the selected extra columns in a single run.
Effect on the result: Each column's result equals the result of running it alone.
On the validation data: write + read, math, science, socst: the values match those from running them one at a time.
One-sample K-S, other distributions: Uniform, Exponential, Poisson; K-S uniform parameters, K-S exponential mean, K-S Poisson mean, K-S p (parameter from data) (Monte Carlo (SPSS 27), Asymptotic (as if the parameter were known)), K-S Monte Carlo samples, K-S Monte Carlo seed
Concept: The one-sample Kolmogorov-Smirnov test can compare the observed cumulative distribution with theoretical distributions other than the normal: the uniform, where every value is equally likely; the exponential, a typical model for waiting times between events; and the Poisson, for counts of events in a fixed period. When the distribution's parameters are estimated from the data, the classical p value is too optimistic, so p can be computed by Monte Carlo simulation instead.
Everyday example: If the bakery manager thinks morning customers arrive at random, the number arriving every 5 minutes should follow a Poisson distribution and the time between two customers an exponential one. The K-S test says whether this expectation fits the data; if not, perhaps customers arrive in groups off a shuttle bus.
What it does: Tests the sample against a uniform, exponential or Poisson distribution. Blank parameters are estimated from the data. Gives D, D+, D-, Z = √n·D; p is Monte Carlo (default 10000 samples, seed 2000000, 99% CI) or asymptotic. It does not enter the overall normality decision.
Effect on the result: Only a separate output section is added. With the parameter estimated from the data the asymptotic p is conservative (Lilliefors effect); therefore Monte Carlo is the default.
On the validation data: faithful$eruptions (uniform, a = 1.6, b = 5.1): D = 0.2005, Z = 3.3067307. aircondit (exponential, n = 12): Monte Carlo p 0.5298 [0.5169, 0.5427]; asymptotic p 0.7939.
Example on real data
Data: UCLA OARC hsb2 (High School and Beyond, n = 200)
Variables: column write (writing score)
- Question: Could the students' writing score (write, n = 200) come from a normal distribution?
- Result: All three tests reject normality: Shapiro-Wilk W = 0.9470, Kolmogorov-Smirnov (Lilliefors) D = 0.1343, D'Agostino-Pearson K² = 18.99; p < .001 in all three. Overall verdict: not normal.
- Source: Skewness = -0.482 (SE = 0.172, z = -2.80) and kurtosis = -0.750 (SE = 0.342, z = -2.19); the distribution is slightly left-skewed and flat.
- Interpretation: The departure is moderate; in a large sample (n = 200) normality tests flag even small departures. The decision should be made together with the Q-Q plot and the size of skewness and kurtosis.
- Next step: With large n, t-tests and ANOVA are robust for comparing means; for small groups nonparametric counterparts (Mann-Whitney U, Wilcoxon, Kruskal-Wallis, Friedman) should be considered.
- APA: A Shapiro-Wilk test showed that writing scores were not normally distributed, W = 0.947, p < .001.
Related concepts
Q-Q and P-P plotsNormality assumptionSignificance level (α)Anderson-Darling testHistogram and bin countKolmogorov-Smirnov test and Lilliefors correctionD'Agostino's K² testBox plot
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.