Statistics Glossary → MR × Categorical Crosstab
MR × Categorical Crosstabanalysis
An analysis of whether the rate at which each option of a multiple-response question is chosen differs between the groups of a single-response categorical variable (such as sex, age group or education level).
Concept
Because a person may tick several options in a multiple-response question, the answers cannot be crossed with a grouping variable as if they were a single categorical variable: the total number of responses exceeds the number of people, which breaks the independence assumption of the ordinary chi-square test. This analysis gets round the problem by treating each option separately: for every option a 'selected / not selected × group' table is built.
In each option's table the selection rates of the groups are compared with a chi-square test, answering questions such as 'does the share choosing coffee vary by age group?'. Since as many tests are run as there are options, the risk of false alarms grows; Bonferroni, Holm or Benjamini-Hochberg adjustments control this risk. The multiple marginal independence (MMI) test takes all options together and gives a single answer to 'is there any association between group and option preferences?'
Each person is counted only once in each option's table, so the individual tests are valid; but since the options come from the same people, the tests are not independent of one another. For a significant option, group-pair z-tests show which groups differ. If the groups are small, Fisher's exact test should be consulted instead of the chi-square approximation.
When to use it
- To test whether the answers to a multiple-response survey question differ by a grouping variable such as sex, age group, region or education level.
- If the question allows only one answer, use an ordinary crosstab and the chi-square test of independence; if only the overall distribution of options is needed, MR frequency analysis is enough.
- To study the joint distribution of two multiple-response questions, use the MR × MR crosstab.
An everyday example
The café splits the answers of its 200 customers to 'which drinks do you choose?' into three age groups (18-29, 30-49, 50 and over). Tea is ticked at a similar rate at every age (70-80%), whereas coffee is chosen by 80% of the young but only 40% of the over-50s. The analysis runs a separate chi-square test for each drink; the adjusted p values show that coffee is the drink linked to age, while tea preference does not change with age.
Parameters in MerQur
MR Set
Concept: The multiple-response question set to be compared across groups. Each of its options is treated as a separate 'selected / not selected' variable, and a separate table with the grouping variable is built for each.
Everyday example: In the café example the set holds four drinks (tea, coffee, ayran, fruit juice); the analysis builds four separate 'chose this drink? × age group' tables.
What it does: The multiple-response question set (same setup paths as the MR Frequency analysis).
Effect on the result: For each option a 2 x G table (selected/not selected x group) is built.
On the validation data: The Kent binary, text and slot forms give the same result. In the farmer1 slot form all 5 of the 5 sources are present (including Veterinarian and Professional consultant).
Grouping variable
Concept: The single-response variable in which each person belongs to exactly one category; it forms the columns of the tables. As the number of groups grows, each test's degrees of freedom (number of groups − 1) rise and fewer people fall into each group.
Everyday example: In the café example the grouping variable is age group with three levels. Each customer is in exactly one of them; customers with missing age are left out.
What it does: The single-response categorical variable (SRCV). Cases with a missing group value are removed.
Effect on the result: Determines the columns (groups) and the degrees of freedom of each χ² (G - 1). The group list does not contain the set's own columns; if a column of the set is given, a clear error message is written.
On the validation data: For Gender, 200 and 214 cases (21 missing removed); for Ed, 88/16/31/113/14.
p adjustment (No adjustment, Bonferroni, Holm, Benjamini-Hochberg (FDR))
Concept: Because a separate test is run for each option, the chance that at least one comes out significant by luck grows even when there is no real difference. Bonferroni and Holm limit the probability of at least one false alarm across all tests (Holm is more powerful); Benjamini-Hochberg controls the share of false alarms among the findings declared significant (the false discovery rate).
Everyday example: Had the café asked about 20 drinks, at the 5% level about one drink would come out 'significant' by chance even with no link to age at all. The adjustment makes sure that coffee's link with age is not that kind of fluke.
What it does: Applies a multiple-comparison adjustment to the p-values of the per-option χ² tests; the "significant" decision is made with the adjusted p. Holm is more powerful than Bonferroni; BH controls the false discovery rate.
Effect on the result: Only the adjusted p and the decision change.
On the validation data: farmer1 Y5 (raw p = 0.02715): Bonferroni 0.1357, Holm 0.1357, BH 0.1357; Y2 (0.0956): 0.4778 / 0.3823 / 0.2389. By default Y5 is significant; with none of the adjustments is it.
% of cases base (Cases with at least one selection, All valid cases)
Concept: Sets the base of the within-group percentages of cases: those in the group who ticked at least one option, or all valid people in the group including those who ticked nothing. It changes only the descriptive percentages; the tests use everyone in the group.
Everyday example: If 5 of the 60 customers over 50 ticked no drink, the 24 who chose coffee are 44% of the 55 who ticked something but 40% of all 60.
What it does: Determines the denominator of the within-group % of cases: "Cases with at least one selection" (default) or "All valid cases" (including those who selected nothing; MRCV and χ² base).
Effect on the result: Only the % of cases table and the chart change; χ², Fisher, z tests and MMI are computed in both options with all cases in the group.
On the validation data: Kent Laptop: default 90.9% / 94.3% (base 197 / 211), "all cases" 89.5% / 93.0% (base 200 / 214). In farmer1 there is no difference because everyone selected at least one source.
Confidence Level (%90, %95, %99, Other (custom))
Concept: No confidence intervals are given in this analysis; the chosen level only sets the threshold for the 'significant' decision (α = 1 − confidence level).
Everyday example: If the café only wants to act on very strong differences, it chooses 99% confidence; drinks whose adjusted p falls below 0.01 are then counted as linked to age.
What it does: In this analysis it is used only as the decision threshold alpha = 1 - confidence level (there is no confidence interval).
Effect on the result: The significance decision, the group-pair letters and the on-screen threshold texts (e.g. "all p ≥ 0.1") follow the selected alpha.
On the validation data: farmer1 Y2 (p = 0.0956) is significant at 90% and not at 95%; Y5 (0.0271) is not significant at 99%. At 80%, groups 1-3 differ in Y1 and 1-4 in Y2.
Additional tests: Group-pair z-tests
Concept: When an option shows an overall difference between groups, shows which pairs of groups the difference comes from. The selection rates of every pair of groups are compared, adjusted for multiple comparisons, and groups that do not differ share a letter.
Everyday example: Once the overall test for coffee is significant, the z-tests might show that the 18-29 and 30-49 groups do not differ (same letter) while the over-50s choose coffee less than both.
What it does: In each option, compares the selection proportions of two groups with a pooled z test; the Bonferroni adjustment is p x number of pairs; gives a letter display.
Effect on the result: An additional table and letters; other outputs do not change.
On the validation data: farmer1 Y5: Ed 1 (45.5%) and Ed 4 (25.7%) differ, Bonferroni p = 0.0337 (10 pairs); letters 1:a, 2:ab, 3:ab, 4:b, 5:ab.
Additional tests: Fisher's exact test
Concept: Gives Fisher's test, based on exact probabilities, for each option's table instead of the chi-square approximation (exact with two groups, by Monte Carlo with more). It is more reliable when groups are small or an option is rarely chosen.
Everyday example: If only 3 customers over 50 chose fruit juice, the expected count in that cell is small and the chi-square approximation weakens. Fisher's test gives a more reliable p for that drink.
What it does: Exact Fisher in a 2x2 table; Monte Carlo Fisher with fixed margins in a 2xk table (B = 10000, seed 42).
Effect on the result: Additional rows; the main χ² test does not change.
On the validation data: Kent (2x2) exact p: Laptop 0.22557, Phone 0.43527. farmer1 (2x5) Monte Carlo: Y5 0.0236 (exact Fisher-Freeman-Halton 0.0246), Y2 0.0881 (exact 0.0879); the difference is within 3 times the Monte Carlo standard error.
Additional tests: Overall MMI test
Concept: Takes all options together and tests in one go whether the grouping variable is associated with any of the option preferences. It gives an overall 'is there a difference?' answer before the option-by-option tests.
Everyday example: The café first asks 'does age group affect drink preferences overall?'. If the MMI test is significant, the option-by-option tests show which drink drives the difference; if it is not, hunting for differences drink by drink increases the risk of false alarms.
What it does: The Bilder and Loughin (2004) multiple marginal independence test: X²_S = sum of the option χ² values; Bonferroni p = min(1, I x smallest p). It gives a single overall decision for all options.
Effect on the result: Additional rows. The Rao-Scott and bootstrap versions of MRCV are not in MerQur (documented in the interface).
On the validation data: farmer1 X²_S = 30.836, Bonferroni p = 0.13573; Kent X²_S = 2.922, p = 0.83126 (Bonferroni adjusted, without an added constant).
Example on real data
Data: Loughin and Scherer (1998), R MRCV farmer1
Variables: multiple response set kaynak (veterinary information sources), group Ed (education level, 5 groups)
- Question: Do farmers' preferences for veterinary information sources (multiple response: feed companies, veterinarian, extension service, magazines, consultant) differ by education level (Ed, 5 groups)?
- Method: For each option a separate Pearson χ² is computed on a 2 x 5 (selected / not selected x education) table; the % of cases base is the farmers in the group with at least one selection (n = 262).
- Result: Only the "feed companies and representatives" preference differs significantly: χ²(4) = 10.95, p = .027; for the other four sources p > .05 (veterinarian p = .096, extension service p = .329, magazines p = .841, consultant p = .202).
- Effect size: For feed companies Cramér V = 0.204 (small to medium); for the other sources V is between 0.07 and 0.17.
- Descriptive: About 45% of education groups 1 and 3 and about 25% of groups 2 and 4 chose feed companies; in all groups the most frequent source is magazines (43-55%).
- Limitation: Three sources have cells with expected frequency below 5; since five separate tests are run, a multiple-comparison adjustment is needed (Bonferroni-adjusted overall MMI test: p = .136; with this adjustment the single significant result also disappears).
- APA: The proportion choosing feed companies as an information source differed by education level, χ²(4, N = 262) = 10.95, p = .027, V = .20.
Related concepts
Post hoc tests (pairwise comparisons)Multiple marginal independence tests (MMI, SPMI)Fisher's exact testMultiple-response set (MR set)Percent of cases and percent of responsesMultiple comparisons and the familywise error rate (FWER)Exact test and exact p-valueBonferroni correction
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.