MerQuR+Statistics Glossary

Statistics Glossary → Runs Test

Runs Testanalysis

A nonparametric test of whether the observations in a sequence arrive in random order, or whether similar values cluster together or alternate in a regular way.

Concept

For data collected over time or in a fixed order, independence of the observations is a core assumption of many analyses. The runs test asks whether the order of the values is random: do high values come one after another (clustering), or do high and low values keep switching (alternation)?

Each observation is first labelled 'above' or 'below' a cut point, usually the median. Every unbroken stretch of the same label is a run; the sequence A A B A B B B, for instance, has 4 runs. Under randomness the expected number of runs depends only on the sizes of the two groups: E[R] = 2n₁n₂/(n₁ + n₂) + 1. Clearly fewer runs than expected means similar values cluster together; clearly more means the values alternate systematically.

Because the test only looks at which side of the cut each value falls on, it needs no distributional assumption; for the same reason it ignores magnitudes and does not say how strong a dependence is, as an autocorrelation coefficient would. A significant result shows that the sequence is not random, not why (trend, seasonality, serial dependence).

When to use it

  • To check quickly whether a series of measurements taken in time order, or the residuals of a model, are in random order.
  • If the values are already binary (0/1, heads/tails, defective/non-defective), it tests the randomness of the sequence directly.
  • To see the strength and lag of serial dependence, the autocorrelation function or the Ljung-Box test is more informative; for first-order autocorrelation in regression residuals, the Durbin-Watson test.

An everyday example

A bakery weighs its loaves in order every morning. Over 30 days, days above the median are marked A and days below it B. If the record moves in long blocks such as 'A A A A B B B B B A A A…', there may be only 8 runs, whereas a random order would give about 16. This suggests the oven setting drifts in the same direction for several days at a time. Had there been 28 runs, as in 'A B A B A B…', it would point instead to a habit of over-correcting each day for the day before.

Parameters in MerQur

Column

Concept: Because the runs test examines the order itself, the row order of the chosen column must be meaningful (usually time order). If the values are already binary, the sequence is taken as already split into two labels.

Everyday example: The bakery's 30 daily loaf weights should sit in one column in date order. Had the records been sorted by weight, the number of runs would be meaningless, because the order would no longer reflect time.

What it does: Selects the sequence whose order is meaningful (time). If the values are only 0 and 1, the sequence is used as is and none of the cut settings has an effect.

Effect on the result: Determines all results.

On the validation data: hsb2 female (0/1): 4 runs (E = 100.19), Z = -13.7496; the cut type, value, and rule do not change the result (by design).

Confidence Level (%90, %95, %99, Other/custom 80.0-99.9)

Concept: Sets the decision threshold (α = 1 − confidence level): if p falls below it, the hypothesis that the sequence is random is rejected. The test statistic itself does not change.

Everyday example: If the bakery manager only wants to send the oven for servicing on very strong evidence, they choose 99% confidence; a sequence with p = 0.03 would then not count as 'non-random', though it would at 95%.

What it does: Used only for the decision threshold (α) and the E[R] ± z·SD band in the plot.

Effect on the result: Z and p do not change; the decision can change.

On the validation data: lh (p = 0.003603): at 99.9% "H₀ accepted (p >= 0.001)", at the other levels the clustering decision.

→ Confidence level and confidence interval

Cut point (Median, Mean, Mode, Custom value) and custom value

Concept: The threshold used to split a continuous measurement into two labels, 'above' and 'below'. The median keeps the two groups roughly equal in size; the mean or a meaningful target value can also be chosen, and the number of runs depends on this choice.

Everyday example: For the bakery, the 500-gram target weight printed on the bag could serve as the cut point instead of the median. The question then becomes 'are the days above and below target in random order?'

What it does: Determines the cut point that splits a continuous sequence into two groups: median, mean, mode (the smallest if there are several), or a custom value. The custom value is read only when "Custom value" is selected.

Effect on the result: Group sizes, the number of runs, Z, and p change.

On the validation data: LEW: median -162 -> 120 runs, Z = 2.6938, p = 0.007065; mean -177.435 -> Z = 2.7005; mode 83 (two modes: 83 and 194) -> Z = 4.0664; custom 0 -> Z = 4.6212.

→ Cut point (dichotomization)

Values equal to the cut (Lower group (x > cut -> upper), Upper group (x >= cut -> upper; SPSS rule))

Concept: The rule that decides which group observations exactly equal to the cut point go into. When values are rounded or discrete there may be many ties, and this rule can change the number of runs, so it should be reported.

Everyday example: If loaves are weighed in 10-gram steps, many days may come out at exactly the 500-gram median. Whether those days count as 'below' or 'above' moves the boundaries of the runs and so changes the run count.

What it does: Determines which group observations with x = cut go to.

Effect on the result: Without ties at the cut (LEW) the two rules are the same; in discrete data the result can change markedly and should be stated in the report.

On the validation data: discoveries (20 observations equal to the median 3): lower rule 33 / 67, 44 runs, Z = -0.2777, p = 0.781; upper rule 53 / 47, 38 runs, Z = -2.5865, p = 0.0097.

→ Cut point (dichotomization)

Hypothesis direction (Two-sided, Fewer runs (clustering), More runs (alternation))

Concept: States which departure from randomness is of interest: a two-sided test looks at both too few and too many runs; a one-sided test looks only for clustering (fewer runs) or only for alternation (more runs). The direction should be chosen before looking at the data.

Everyday example: If the oven technician is worried only about the setting drifting the same way for several days, they choose 'fewer runs'. If both drift and over-correction are of interest, the two-sided test is appropriate.

What it does: One-sided test in the direction of fewer or more runs.

Effect on the result: When a direction is selected, the main p and the decision are based on the directional p; the two-sided p is shown in a separate row.

On the validation data: LEW "Fewer runs": directional p = 0.996468, decision H₀ not rejected; "More runs" p = 0.003532, significant. lh "Fewer runs" p = 0.001802 significant. The two-sided p is 0.007065.

→ One-sided and two-sided tests (hypothesis direction)

Continuity-corrected Z (checkbox)

Concept: The number of runs is a whole number, but the Z statistic compares it with a continuous normal distribution. The continuity correction shrinks the difference by half a unit to reduce this mismatch, giving a more cautious p in short sequences.

Everyday example: If the bakery has only 20 days of records, the step from 9 to 10 runs can look larger under the normal approximation than it really is. The corrected Z guards against an over-optimistic p in such a small sample.

What it does: Z and p with |R - E| reduced by 0.5 (Z = 0 if |R - E| <= 0.5).

Effect on the result: Extra row only.

On the validation data: LEW Zcc = 2.6229, p = 0.008719; lh Zcc = -2.7648, p = 0.005696.

→ Continuity correction (Yates)

Exact p (checkbox)

Concept: Computes the p value from the exact probability distribution of the number of runs for the given group sizes instead of the normal approximation; it is more reliable especially for short sequences.

Everyday example: If the bakery must decide with only two weeks of records, the normal approximation is rough. The exact p counts directly how often, among all possible orderings of the 14 days, a run count at least as extreme as the observed one would occur.

What it does: The exact p from the Swed-Eisenhart runs distribution (two-sided: |r - E| >= |R - E|).

Effect on the result: Extra row only; the decision row uses the asymptotic p.

On the validation data: LEW 0.008562, lh 0.004870, discoveries 0.820972, hsb2 write 0.886747.

→ Exact test and exact p-value

Example on real data

Data: lh hormone series, Diggle 1990, R datasets

Variables: column value (luteinizing hormone level measured at 10-minute intervals, 48 observations)

  • Question: Is the LH hormone series of 48 consecutive measurements random? The runs test compares the number of runs (consecutive blocks with the same sign) in the above/below median sequence with the expected number.
  • Descriptive: Median 2.3; 23 values are above the median, 25 are below or equal. The observed number of runs is 15, the expected number under randomness is 24.96 (SD 3.42).
  • Result: Z = -2.91, p = 0.0036. The null hypothesis is rejected: the sequence is not random.
  • Interpretation: Because there are fewer runs than expected, similar values follow one another (clustering, positive autocorrelation); in time series analysis the independence assumption is not met.
  • Limitation: The group to which values equal to the median are assigned changes the result; the "Lower group" and "Upper group" options give different Z values. The rule should be stated in the report.
  • APA: Runs test, R = 15, E[R] = 24.96, Z = -2.91, p = .004.

Related concepts

Significance level (α)Confidence level and confidence intervalAsymptotic (normal) approximationCut point (dichotomization)Continuity correction (Yates)One-sided and two-sided tests (hypothesis direction)Exact test and exact p-valueTies and zero differences (rank tests)

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.