MerQuR+Statistics Glossary

Statistics Glossary → MR Frequency Analysis

MR Frequency Analysisanalysis

An analysis summarising, for a multiple-response question where a person may tick more than one option, how many times each option was chosen and what percentage of respondents chose it.

Concept

In 'tick all that apply' questions one person can give several answers, so an ordinary frequency table will not do: the percentages add up to more than 100% and no single variable can hold the information. The multiple-response question is treated as a set (MR set), in which each option is kept as its own 'selected / not selected' column or as delimited text.

The analysis gives two different percentages. The percentage of responses is an option's share of all ticks and sums to 100% across options. The percentage of cases shows what percentage of respondents ticked that option; because one person may tick several options, these percentages can sum to more than 100%. Dividing the total number of responses by the number of people gives the average number of options ticked per person.

For most research questions it is the percentage of cases that is interpreted: '75% of respondents prefer tea'. Because the base of the percentage of cases (those who made at least one selection, or all respondents) can change the result, it should be stated in the report. Since percentages of cases are proportions, giving them with confidence intervals shows the sampling uncertainty; note, though, that option percentages are not independent of one another, because one person contributes to several options.

When to use it

  • To describe the distribution of a survey question in which more than one option may be ticked (e.g. devices used, information sources, reasons for complaint).
  • If each person can tick only one option, an ordinary frequency table is enough.
  • To ask whether multiple-response answers differ by a grouping variable use the MR × categorical crosstab; for the joint distribution of two multiple-response questions use the MR × MR crosstab.

An everyday example

A café asks 200 customers 'Which hot and cold drinks do you regularly choose? (You may tick more than one)'. Tea is ticked 150 times, coffee 120, ayran 70 and fruit juice 60; there are 400 ticks in all, an average of 2 drinks per customer. Tea makes up 37.5% of all ticks (percentage of responses), but 75% of customers ticked tea (percentage of cases). When the café says 'three quarters of our customers drink tea', it is using the percentage of cases.

Parameters in MerQur

MR Set

Concept: The set that brings all options of a multiple-response question together as one question. The options may be stored as separate 'selected / not selected' columns, as delimited text in one cell, or as response slots each holding an option name; for the analysis they are all turned into the same logic.

Everyday example: In the café survey there might be four separate 0/1 columns for tea, coffee, ayran and fruit juice, or a single column with text such as 'tea, coffee'. The set defines this information as one question called 'drink preference'.

What it does: Selects the multiple-response question set to be analyzed. The set is built in the Data > Multiple Response panel by grouping binary columns, by parsing a delimited text column, or by grouping slot columns that carry category names.

Effect on the result: The coding (binary or text) is detected automatically from the first 50 non-empty cells; option names come from the definition of the set. In slot form, all unique non-empty values become options.

On the validation data: The Kent data give the same result in all three forms (binary, text, slot): 429 respondents, 991 responses, Laptop 397 (40.1% of responses, 92.5% of cases). In farmer1 slot form all 5 of the 5 sources are counted (Veterinarian 90, Professional consultant 44).

→ Multiple-response set (MR set)

Counted value

Concept: When options are stored in separate binary columns, states which value means 'selected'. The coding may be 1/2 or Yes/No instead of 0/1; if the wrong value is chosen, 'not selected' answers are counted as selections.

Everyday example: If the café survey is coded 1 = yes, 2 = no, the counted value must be 1. Choosing 2 by mistake would count the 50 customers who do not drink tea as tea drinkers and turn the results upside down.

What it does: Determines the value counted as "selected" in binary coding (the counted value of the dichotomy group). If empty, 0/1 is detected automatically.

Effect on the result: Applies only to a binary (dichotomous) coded set (0/1, 1/2 or at most two distinct text values across the set, e.g. Yes/No). In a text or slot set that carries option names it is ignored and a warning is written; when the base is 0, percentages are written as "-" (undefined).

On the validation data: In farmer1, "1" equals the default; "0" gives the number of non-selections: 218 for Y1 (base 250 cases: 87.2%; 83.2% on the all-cases base). In Kent "0" counts nothing (blanks are missing): 0 respondents, percentages "-".

% of cases base (Cases with at least one selection, All valid cases)

Concept: Sets the base of the percentage of cases: people who ticked at least one option, or all valid respondents who saw the question. If some people ticked nothing, the two bases give different percentages; which one was used should be stated in the report.

Everyday example: Suppose 10 of the café's 200 customers left the question blank. The 150 who ticked tea are 79% of the 190 who ticked something, but 75% of all 200 customers.

What it does: Determines the denominator of % of cases: those who checked at least one option or all valid cases.

Effect on the result: Only % of cases and the confidence interval change; frequency and % of responses do not change.

On the validation data: Kent Laptop 92.54% (n = 429) -> 91.26% (n = 435); farmer2 Lagoon 51.44% (n = 278) -> 51.25% (n = 279). In farmer1 there is no difference because everyone selected at least one source.

→ Percent of cases and percent of responses

Missing data (Missing = not selected, Exclude partially missing cases)

Concept: Sets how blank cells are interpreted: a blank option can be counted as 'not selected', or a person with missing answers can be removed from the analysis. In multiple-response questions a blank usually means 'I did not tick it', but gaps from data entry may mean something else.

Everyday example: On a paper survey where only ticked boxes were filled in, blank boxes mean 'not selected'. But if the back page of the survey was never filled in at all, those people's answers are genuinely missing and excluding them may be more appropriate.

What it does: Chooses between counting a blank cell as "not selected" (default) and excluding the case.

Effect on the result: In a binary set, a case with any blank column is removed; in slot and text sets, only a case with all slots blank is removed.

On the validation data: In Kent (1/blank coding) in the binary set, 12 of 435 cases remain and all devices become 100%; since a blank means "does not own" in this coding, the option gives a meaningless result. In the slot set 6 cases are excluded.

→ Listwise and pairwise deletion

Proportion CI (None, Wilson, Clopper-Pearson) and Confidence Level (90%, 95%, 99%, Other (custom))

Concept: Percentages of cases are proportions computed from a sample; the confidence interval shows the range in which the true proportion may lie. Wilson and Clopper-Pearson intervals are more reliable than the simple formula in small samples or for proportions near 0 or 1.

Everyday example: If 75% of customers ticked tea, with a sample of 200 the 95% confidence interval runs from about 69% to 80%. Had only 20 customers been asked, the interval would be much wider.

What it does: Adds the Wilson score or Clopper-Pearson exact binomial confidence interval for % of cases proportions; the level is between 80.0 and 99.9. For small n or extreme proportions, Clopper-Pearson (conservative) or Wilson (Brown, Cai and DasGupta, 2001) is recommended.

Effect on the result: Only the interval column is added; if no interval is selected, the confidence level changes no output.

On the validation data: Kent Laptop (397/429): Wilson 95% [89.66, 94.67], 90% [90.18, 94.37]; Clopper-Pearson 95% [89.63, 94.84], 99% [88.66, 95.44]; Other (40/429) Wilson [6.92, 12.45].

→ Confidence interval for a proportion (Wilson, Clopper-Pearson, Agresti-Coull, Jeffreys)

Example on real data

Data: Kent State University Libraries SPSS tutorial data (device ownership), SPSS Multiple Response tutorial

Variables: multiple response set devices (comma-separated devices: Laptop, Phone, Tablet, Other)

  • Question: What is the distribution of devices owned by the students (laptop, phone, tablet, other)? Each student can check more than one device.
  • Result: 429 of the 435 cases checked at least one device; 991 responses in total, a mean of 2.31 devices per person.
  • Descriptive: Laptop 397 (40.1% of responses, 92.5% of students), phone 386 (39.0%; 90.0%), tablet 168 (17.0%; 39.2%), other 40 (4.0%; 9.3%).
  • Interpretation: The % of responses column gives the share of responses, the % of cases column answers "what percentage of students own this device"; the sum of % of cases exceeds 100% (231) because a student checks more than one option.
  • Limitation: This analysis is descriptive; Cochran Q tests the equality of option proportions and a multiple-response crosstab compares groups.
  • APA: 92.5% of the students owned a laptop, 90.0% a phone and 39.2% a tablet (n = 429; 991 responses).

Related concepts

Confidence level and confidence intervalMultiple-response set (MR set)Percent of cases and percent of responsesConfidence interval for a proportion (Wilson, Clopper-Pearson, Agresti-Coull, Jeffreys)Listwise and pairwise deletionExact test and exact p-value

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.