MerQuR+Statistics Glossary

Statistics Glossary → Chi-Square Independence

Chi-Square Independenceanalysis

A test of whether two categorical variables are associated, comparing the observed counts in a crosstab with the counts expected if there were no association.

Concept

When two categorical variables are summarised in a crosstab (e.g. vaccination status × falling ill), the real question is whether the distribution of one variable changes across the levels of the other. If it does not, the variables are independent; if it does, they are associated.

If independence holds, the expected count in each cell is E = row total × column total / n. The test adds up the differences between observed (O) and expected (E) counts: χ² = Σ (O − E)² / E. The bigger the differences, the bigger χ²; the p value comes from a chi-square distribution with (r − 1)(c − 1) degrees of freedom. The likelihood-ratio G² answers the same question with a slightly different yardstick and gives similar results in large samples.

Observations must be independent (each person appears in only one cell) and the expected counts must not be too small; a common rule is that at least 80% of cells should have an expected count of 5 or more. A significant χ² only says that an association exists: for its strength read effect sizes such as Cramér's V, for which cells drive it read the residuals, and for its direction and size in 2x2 tables read the relative risk or odds ratio. In very large samples even practically trivial associations can be significant.

When to use it

  • To test whether two categorical variables are associated in a sample of separate people or units (e.g. treatment group × recovery, gender × preference).
  • When expected counts are small, especially in 2x2 tables, Fisher's exact test or an exact / Monte Carlo p is preferred.
  • If the same people were measured twice (before/after) use the McNemar test; to examine the association within strata of a third variable use the Cochran-Mantel-Haenszel test; for interactions among three or more variables use a log-linear model.

An everyday example

Suppose a school follows 200 vaccinated and 200 unvaccinated pupils through flu season. Of the vaccinated, 30 (15%) catch flu; of the unvaccinated, 50 (25%). If vaccination and illness were unrelated, both groups would be expected to show the overall rate of 20%, that is 40 cases each. The chi-square test gives the probability that the observed 30 and 50 would stray this far from the expected 40 by chance, and the relative risk shows that the vaccinated pupils' risk of flu is 0.6 times that of the unvaccinated.

Parameters in MerQur

Column 1 and Column 2

Concept: The two categorical variables that form the rows and columns of the crosstab. The chi-square statistic is unaffected by which variable goes in the rows; directional measures, where one variable 'explains' the other, and ordinal measures do depend on this choice and on the order of the levels.

Everyday example: In the school example, vaccination status (vaccinated / unvaccinated) goes in the rows and flu (caught / not caught) in the columns. Swapping them leaves χ² unchanged, but reverses a directional question such as 'how much does knowing vaccination status help predict flu?'

What it does: Selects the row and column variables of the cross-table. Numeric levels are sorted ascending, strings in code point (ASCII) order; the level order can also be specified. In a summary (frequency) table, factor columns can also be selected.

Effect on the result: X², G², V, C, and the symmetric PRE are unaffected by swapping; directional measures (Somers d, lambda, GK tau, U) change direction; ordinal measures and the linear-by-linear test depend on the level order.

On the validation data: Pain data Somers d C|R = 0.4427, R|C = 0.2569; swapping exchanges the two values. JobSat with the level order specified gives gamma 0.2211009; with the level order empty, alphabetical order and an explicit warning are given.

Confidence Level (%90, %95, %99, Other/custom 80.0-99.9)

Concept: Sets the decision threshold (α = 1 − confidence level) and the width of the confidence intervals for estimates such as the relative risk, odds ratio and Cramér's V. The test statistic and p value do not change.

Everyday example: With a relative risk of 0.6, the 95% confidence interval might run from about 0.40 to 0.91; at 99% confidence the interval widens and may approach or even include 1.

What it does: α = 1 - confidence level. The decision rule (p < α) and all CIs (2x2 OR, RR, RD, ordinal measures, Cramér V, Monte Carlo p) are computed at this level.

Effect on the result: The test statistic and p do not change; the CI width and the decision change. Cramér V and ordinal measure CIs are clipped to [-1, 1] ([0, 1] for V).

On the validation data: FatComp data (Yates p = 0.0742) CI for OR = 8.25: 80% [2.2792, 29.8627], 90% [1.5827, 43.0037], 95% [1.1535, 59.0029], 99% [0.6217, 109.4843]. The decision is "rejected" at 90% and 80%.

→ Confidence level and confidence interval

φ, 2x2 (checkbox)

Concept: An effect size for the strength of association in 2x2 tables; it corresponds to the correlation between two binary variables. 0 means no association; values approaching 1 mean a strong one.

Everyday example: In the school example, with χ² = 6.25 and n = 400, φ = √(6.25/400) ≈ 0.13: the link between vaccination and flu may be significant, but it is weak, because some vaccinated pupils also caught flu.

What it does: phi = √(X²/n), from the uncorrected Pearson X²; written only in a 2x2 table (in RxC a "2x2 only" note is given).

Effect on the result: Extra effect size row only.

On the validation data: hsb2 schtyp x female 0.0153; aspirin 0.0337.

→ Phi (φ) coefficient

C, contingency coefficient (checkbox)

Concept: An older measure of association that can be applied to tables of any size. Because its maximum depends on the table size (about 0.71 at most in a 2x2 table), comparing tables is difficult, so Cramér's V is usually preferred.

Everyday example: In the school example C ≈ 0.12. If the same school studied another relationship in a 3x3 table, C's upper limit would differ, so the two values could not be set side by side directly.

What it does: C = √(X²/(X² + n)).

Effect on the result: Extra row only. Because its upper bound depends on the table size, V is preferred.

On the validation data: Color data 0.1635; GSS 0.0843.

→ Contingency coefficient C

Yates correction, 2x2 (checkbox; default on)

Concept: A correction for 2x2 tables that subtracts half a unit from each difference to reduce the optimism of comparing whole-number counts with a continuous chi-square distribution. It increases the p value and makes the test more cautious; some authors consider it overly cautious.

Everyday example: If the school example were small, say two classes of 20, the uncorrected test might be borderline significant and the Yates-corrected test not. As the sample grows, the two results converge.

What it does: In 2x2 the main statistic is the continuity-corrected X²; the effect sizes are always computed from the uncorrected X². No effect in RxC.

Effect on the result: X², p, and the decision.

On the validation data: FatComp: on 3.1879, p = 0.0742 (not rejected); off 4.9597, p = 0.0259 (rejected). hsb2 schtyp x female: on 0.0005, p = 0.9815; off 0.0470, p = 0.8283.

→ Continuity correction (Yates)

G-test, likelihood ratio (checkbox)

Concept: A statistic based on the likelihood ratio instead of Pearson's chi-square: it sums the distance between observed counts and those expected under independence on a logarithmic scale. In large samples it gives results similar to Pearson's and speaks the same language as log-linear models.

Everyday example: In the school example G² and Pearson's χ² come out almost the same. If the researcher later adds age group and fits a three-way log-linear model, the G² results can be compared directly with that model.

What it does: The main statistic becomes G² = 2 Σ O ln(O/E). In 2x2, if Yates is also on, the continuity correction is applied to G² as well; the APA line says "(Yates-corrected)".

Effect on the result: The main statistic, p, and the decision are based on G².

On the validation data: hsb2 female x ses G² = 4.6789; FatComp 3.2290 with Yates on, 5.0975 with Yates off.

→ G-test (likelihood-ratio chi-square, G²)

Expected counts (checkbox)

Concept: A table showing how many observations each cell would hold if the two variables were independent. It shows the logic of the test and lets you check whether the expected counts are small, that is, whether the chi-square approximation is valid.

Everyday example: In the school example the four expected counts are 40, 160, 40 and 160. All are well above 5, so the chi-square approximation can be trusted.

What it does: The table E = row total x column total / n.

Effect on the result: Extra table only.

On the validation data: In the Color data the expected values are given for all 15 cells; the expected values are also computed for the GSS and Agresti Table 3.2 data.

→ Expected frequency

Standardized residuals (checkbox)

Concept: The difference between observed and expected counts in each cell, divided by the square root of the expected count. It shows which cells contribute most to the overall χ²; values beyond about 2 stand out.

Everyday example: In the unvaccinated-and-flu cell, with 50 observed and 40 expected, the residual is (50 − 40)/√40 ≈ 1.58. That cell has more cases than expected, but is not extreme on its own.

What it does: Pearson residual (O - E)/√E; |r| > 2 is marked.

Effect on the result: Extra table only.

On the validation data: In the Color data the standardized residuals of the 15 cells are given (e.g., green-black -2.40).

→ Standardized and adjusted residuals (contingency tables)

Adjusted residuals (checkbox)

Concept: A version of the standardized residual that also accounts for the row and column totals; under independence it is approximately standard normal. Values beyond ±1.96 therefore indicate that a cell departs significantly from expectation.

Everyday example: In the same school example the adjusted residual is about ±2.5 in each cell. Even the cell whose simple residual was only 1.58 holds significantly more cases than expected once the margins are taken into account.

What it does: Haberman adjusted standardized residual; |z| > 1.96 is marked.

Effect on the result: Extra table only.

On the validation data: Agresti Table 3.2: (<HS, Fund) 4.5349, (<HS, Mod) -2.5522, (Bachelor, Fund) -6.8098, (Bachelor, Lib) 6.2525.

→ Standardized and adjusted residuals (contingency tables)

All tests table (checkbox)

Concept: Shows the different tests that can be applied to the same table side by side: Pearson, Yates, likelihood ratio, Fisher and, when levels are ordered, the linear-by-linear association test, which looks for a linear trend. It shows at a glance whether the conclusion depends on the test chosen.

Everyday example: If a reviewer asks 'would Fisher's test lead to the same conclusion?', the table answers at once. Had the school used 'number of vaccine doses' (0, 1, 2) instead of vaccination status, the linear-by-linear test would ask whether the flu rate falls steadily as doses increase.

What it does: Gives Pearson, Yates (2x2), G², Fisher (2x2), and the linear-by-linear M² = (N - 1)·r² test in one table. For M² the numeric level value is used as the score.

Effect on the result: Extra table only.

On the validation data: Agresti infant data (alcohol 0, 0.5, 1.5, 4, 7): M² = 6.5699, p = 0.0104; the scores are the level values, and equally spaced scores give a different M².

Exact / Monte Carlo p (None, Monte Carlo χ², Fisher)

Concept: Computes the p value directly instead of using the chi-square approximation. The exact test sums the probabilities of tables at least as extreme as the observed one with the row and column totals fixed; the Monte Carlo method generates thousands of random tables with the same totals and counts how many are as extreme. These are preferred when expected counts are small.

Everyday example: Had the school followed only 12 vaccinated and 10 unvaccinated pupils, some cells would have expected counts below 5. The exact or Monte Carlo p should then be relied on instead of the chi-square approximation.

What it does: "Monte Carlo χ²": B = 10000 tables with fixed margins, p = (1 + b)/(B + 1), seed 12345. "Fisher": exact Fisher in 2x2, Fisher-Freeman-Halton Monte Carlo in RxC.

Effect on the result: The selected p is used for the main p, the decision, and the APA line.

On the validation data: hsb2 schtyp x race: Fisher MC p = 0.5923 [0.5826, 0.6020]. FatComp Fisher exact p = 0.039305; MC χ² p = 0.0388.

→ Exact test and exact p-value

Weight (frequency) column

Concept: When the data are a summary table with one row per cell rather than raw data with one row per person, this column states how many observations each row stands for. Cell counts are then obtained by summing these weights instead of counting rows one by one.

Everyday example: Instead of entering 400 pupils one by one, the school can prepare a four-row table: 'vaccinated, flu, 30', 'vaccinated, no flu, 170' and so on. With the count column chosen as the weight, the results match those from the 400-row raw data.

What it does: If the rows are a summary table, the cells are sums of this column. Negative or missing weights give an explicit error; with non-integer weights Monte Carlo is skipped; zero-weight levels are dropped.

Effect on the result: All counts and results are computed on the weighted N; "Observations (n)" in the header is the weighted N.

On the validation data: FatComp with Count: X² (Yates) = 3.1879, N = 23; without weights each row counts as 1 observation (X² = 0, p = 1, N = 4). Aspirin N = 22071.

→ Frequency weight (summary-table data)

Column proportions z-test (checkbox)

Concept: Within each row category, compares the proportions in the column groups pair by pair. Because many comparisons are made, the Bonferroni correction controls the risk of false alarms; columns that do not differ from each other share a letter.

Everyday example: Had the school used three columns, unvaccinated, one dose and two doses, the overall χ² would say 'there is a difference' but not where. The column proportions test might show that the flu rate is higher among the unvaccinated than in either dose group, with no difference between one and two doses.

What it does: Two-proportion z test with pooled proportion for the column pairs in each row, Bonferroni (m = C(C - 1)/2), and letter notation.

Effect on the result: Extra table only.

On the validation data: Pain "Yes" row: Dose 0, 1, 2 (b), 3 (ab), 4 (a); the significantly different pairs are the same in all 5 runs; the Bonferroni p values are the uncorrected two-proportion z-test p multiplied by the number of comparisons (m).

→ Column proportions z-test

Ordinal measures (checkbox)

Concept: Coefficients that measure the direction and strength of association when the levels of both variables have a natural order, such as low-medium-high (gamma, Kendall's tau-b and tau-c, Somers' d). They range from −1 to 1; a positive value means that as one variable rises the other tends to rise too.

Everyday example: Had the school crossed number of doses (0, 1, 2) with severity of illness (none, mild, severe), a negative gamma would say that illness tends to be milder as doses increase. Chi-square alone says only that 'there is an association', not which way it runs.

What it does: gamma, tau-b, tau-c, Somers d (C|R, R|C); CI with ASE1, z and p with ASE0. The CIs are clipped to [-1, 1].

Effect on the result: Extra table only; depends on the level order.

On the validation data: Pain data: gamma 0.5313 (ASE 0.0935, CI 0.3480-0.7146), Somers R|C 0.2569, z = 5.151.

→ Ordinal association measures (gamma, tau-b, tau-c, Somers' d)

RR / risk difference, 2x2 (checkbox)

Concept: In 2x2 tables, compares the risk of the event in two groups: the relative risk (RR) shows how many times one group's risk is the other's, and the risk difference shows how far apart they are in percentage points. RR = 1 and a risk difference of 0 mean no difference.

Everyday example: The vaccinated pupils' flu risk is 15%, the unvaccinated pupils' 25%. RR = 0.15/0.25 = 0.6, so the vaccinated risk is 0.6 times the unvaccinated risk; the risk difference is −10 percentage points, about 10 fewer cases per 100 vaccinated pupils.

What it does: The ratio row1 / row2 of the first-column risk (log-Wald CI) and the difference (Wald CI; clipped to [-1, 1]). In RxC a "2x2 only" note is given.

Effect on the result: Extra rows only.

On the validation data: Aspirin RR = 1.8178 [1.4330, 2.3059], RD = 0.0077 [0.0047, 0.0107]. FatComp RR CI [1.1100, 7.1261].

→ Relative risk (RR) and risk difference (RD)

Cramér's V CI + Cohen interpretation (checkbox)

Concept: Cramér's V measures strength of association on a 0-to-1 scale for tables of any size. Being computed from a sample, it has its own uncertainty; the confidence interval shows this uncertainty, and Cohen's thresholds help describe the value in words as small, medium or large.

Everyday example: In the school example V ≈ 0.13 corresponds to a small association. If its confidence interval runs from about 0.03 to 0.22, the true association may be very small or small-to-medium; that is as much as 400 pupils allow us to say.

What it does: The V CI by inverting the noncentral chi-square; an interpretation depending on the degrees of freedom with Cohen (1988) w thresholds (.1 / .3 / .5, divided by √(min(r, c) - 1)). The V label in the header uses the same interpretation.

Effect on the result: Extra row only.

On the validation data: hsb2 female x ses V = 0.1513 [0.0000, 0.2776] (noncentral chi-square CI), "Small"; in a table with V = .4644 the label is "Medium".

→ Cramér's V and Cohen's w

λ, GK τ, uncertainty coefficient (checkbox)

Concept: 'Proportional reduction in error' measures for unordered (nominal) variables: they show by what proportion knowing one variable reduces the error made when predicting the other. 0 means no help and 1 perfect prediction; values can differ depending on which variable is being predicted.

Everyday example: Suppose the school wonders how much knowing a pupil's class helps predict which club they choose (sport, music, chess). If lambda = 0.20, knowing the class cuts prediction errors by 20%.

What it does: Nominal proportional reduction in error (PRE) measures.

Effect on the result: Extra table only; the directional versions depend on the row and column roles.

On the validation data: Pain data lambda C|R 0.1250, R|C 0.2373, symmetric 0.1604; U 0.0515 / 0.1261 / 0.0731; GK tau 0.0412 / 0.1652.

→ Proportional reduction in error measures (lambda, Goodman-Kruskal τ, uncertainty coefficient)

Example on real data

Data: Physicians' Health Study aspirin data, Agresti 2007

Variables: column 1 group (1 = placebo, 2 = aspirin), column 2 mi (1 = myocardial infarction, 2 = none), weight count (summary table frequency)

  • Question: Is aspirin use (placebo and aspirin groups) associated with having a myocardial infarction (MI)? The frequency column (count) is used as the weight; N = 22071.
  • Assumption: The minimum expected count is 146.48; no cell has an expected count below 5, so the chi-square approximation is valid.
  • Result: Yates-corrected χ²(1) = 24.43, p < .001 (uncorrected Pearson χ² = 25.01). The null hypothesis is rejected: group and MI are associated.
  • Effect size: Cramér V = 0.034; the association is statistically significant but very small (a large N makes small differences significant).
  • Additional finding: Odds ratio 1.83, 95% CI [1.44, 2.33]: the odds of MI in the placebo group are about 1.8 times those in the aspirin group; for clinical interpretation the odds ratio is more informative than V.
  • APA: χ²(1, N = 22071) = 24.43 (Yates-corrected), p < .001, V = .03, OR = 1.83, 95% CI [1.44, 2.33].

Related concepts

Standardized and adjusted residuals (contingency tables)G-test (likelihood-ratio chi-square, G²)Ordinal association measures (gamma, tau-b, tau-c, Somers' d)Continuity correction (Yates)Cramér's V and Cohen's wContingency coefficient CColumn proportions z-testProportional reduction in error measures (lambda, Goodman-Kruskal τ, uncertainty coefficient)

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.