Statistics Glossary → Likert Scale Analysis
Likert Scale Analysisanalysis
An analysis that summarises, item by item, how responses are distributed on Likert-type items running from 'strongly disagree' to 'strongly agree', the shares agreeing and disagreeing, and the consistency of the item set.
Concept
Most attitude and satisfaction surveys ask respondents to state how far they agree with a statement using ordered categories (e.g. 1 = strongly disagree … 5 = strongly agree). With such data the first question is what people think on each item: which categories do the answers gather in, what share agree, how many are undecided? Likert scale analysis produces this item-level picture and also checks whether the items together form a consistent scale.
For each item it computes category frequencies and percentages, the sum of the top two categories (top-2, 'agreement'), the sum of the bottom two (bottom-2, 'disagreement') and the middle (neutral) category. Because these percentages are sample estimates they are given with confidence intervals; for proportions the Wilson interval is more reliable than the simple approximation, especially when proportions are close to 0 or 1. Since Likert answers are ordinal, ordinal summaries such as the median and quartiles are reported alongside the mean, and group comparisons use the rank-based Kruskal-Wallis test.
With Likert items it is not certain that the distances between categories are equal, so the mean of a single item should be read with care, together with its distribution and percentages. A scale score obtained by summing many items, on the other hand, usually behaves close to continuous. Answers piling up at one end (a ceiling or floor effect) reduce an item's ability to tell people apart. Negatively worded items must be reverse-coded before they are summarised and summed.
When to use it
- When the results of a survey made up of Likert-type items are to be described at item level, agreement rates reported with confidence intervals, and items compared with one another.
- If the scale's reliability is to be examined in detail (omega, alpha if item deleted, subscales), Cronbach's alpha and reliability analysis is used; if the question is how many dimensions the items fall into, exploratory factor analysis.
- If the question is whether the response distribution on a single item differs between two or more groups, the Mann-Whitney U or Kruskal-Wallis test is suitable; to compare the whole distribution, the chi-square test of independence.
An everyday example
A municipality asks 1,200 passengers ten statements to measure satisfaction with public transport, such as 'Buses arrive on time', 'Stops are clean' and 'Drivers are polite', each answered from 1 to 5. Likert scale analysis shows what percentage of passengers agree with each statement: for example, 78% agree that drivers are polite, but only 41% that buses arrive on time, and 30% are undecided on this item. A diverging bar chart ranks all the statements in one picture, and the municipality can give priority for improvement to the statements with the lowest agreement.
Parameters in MerQur
Items
Concept: The Likert items to be summarised. For each item the response distribution, percentages, confidence intervals and ordinal summaries are given separately; the internal consistency of the scale formed by the selected items together is also summarised with Cronbach's alpha.
Everyday example: The municipality selects the ten public-transport statements. If a non-Likert question such as 'how many minutes is your journey?' were added, the percentages would become meaningless and alpha would drop.
What it does: Gives the descriptives, response distribution, Wilson confidence interval, ordinal summaries and embedded Cronbach's alpha for each item.
Effect on the result: All item-level outputs.
On the validation data: q1 mean 3.133, SD 1.081, skewness 0.059, kurtosis -0.350, median 3, Q1 2.25, Q3 4 (percentiles with the (n+1)p definition); alpha 0.8390.
Scale type (Auto-detect, 5-point, 7-point, Manual range) and Scale range (min-max)
Concept: Defines how many categories the response scale has and its end points; top-2, bottom-2 and neutral percentages and reverse coding are computed from these ends. If one end of the scale was never chosen, automatic detection finds the wrong range, and the range must then be given by hand. Scales with an even number of categories have no neutral middle category.
Everyday example: If no passenger answered 'strongly disagree' (1), automatic detection takes the scale to be 2-5 and computes the 'bottom-2' percentage from categories 2 and 3. Stating by hand that the scale is 1-5 makes the percentages come from the right categories.
What it does: Sets the reported categories, the top-2/bottom-2/neutral percentages and the reverse-coding endpoints. Auto-detect takes the smallest and largest observed integers; with a manual range the given endpoints are used. If the scale endpoints are not observed in the data, "Manual range" should be chosen.
Effect on the result: The category table, the top-2/bottom-2/neutral percentages and reverse coding; "Neutral %" is not shown for an even-numbered scale.
On the validation data: q1-q4 auto 1-5; when 7-point is forced, categories 6 and 7 are 0%. With manual 0-4 category 5 is left out (13.3% for q1). bfi auto 1-6: A2 top-2 68.25%; pisaitems auto 1-4: ST24Q01 top-2 41.00%.
Missing data (Listwise, Pairwise)
Concept: With listwise deletion, a respondent who left even one item blank is removed from all items and all results rest on the same people. With the pairwise method nobody is dropped; each item is summarised from everyone who answered it, so the items' n values may differ.
Everyday example: If 90 of 1,200 passengers left 'stops are clean' blank, listwise deletion summarises every item from 1,110 passengers. With the pairwise method, the other items continue to be summarised from all 1,200 passengers' answers.
What it does: Listwise uses complete rows; pairwise drops no rows, and the denominator of the percentages is all rows.
Effect on the result: The item n and the percentages.
On the validation data: M255 listwise n = 1365; pairwise n = 1428, ITEM13 category 5 55.88% (798/1428).
Confidence Level (%90, %95, %99, custom)
Concept: Each category percentage is a proportion estimated from the sample; the Wilson confidence interval shows the range in which its true value in the population may lie. The higher the confidence level, the wider the interval.
Everyday example: If 41% agree that buses arrive on time with a 95% confidence interval of [38%, 44%], agreement among all passengers can be said to lie roughly in this range. Had only 100 passengers been asked, the interval would be much wider.
What it does: Sets the level of the Wilson confidence interval for a category percentage.
Effect on the result: Only the interval bounds.
On the validation data: For q1 category 3 (43.3%) the 90%, 95%, 99% and 97.5% intervals are computed with the Wilson score method without continuity correction.
Reverse-coded items
Concept: Reverses the scores of items worded in the opposite direction to the characteristic being measured, using the scale's end points, so that a high score points the same way on every item. If reversed items are summarised or summed without recoding, the scale score and item comparisons are misleading.
Everyday example: Agreeing with 'buses are far too crowded' means dissatisfaction. With reverse coding, a high score on this item also corresponds to satisfaction, and the item can be summed in the same direction as the other satisfaction statements.
What it does: Reverses the marked items with x' = scale min + scale max - x (the endpoints come from the selected or specified scale).
Effect on the result: Item statistics, the category distribution and the saved score; alpha is not affected by the shift.
On the validation data: bfi A1 is reversed with 7 - x under manual 1-6 and auto-detect (alpha 0.7038); the A1 mean is 4.588.
Automatic reverse coding (items loading negatively on the first component)
Concept: Infers from the data which items are reversed: it flips items loading negatively on the first principal component, which summarises the items' common tendency. It is for exploration; which items are reversed should be known from the survey design.
Everyday example: If the team that prepared the survey forgot to note the reversed items, automatic reverse coding notices that 'buses are far too crowded' runs opposite to the others. The result must still be checked against the questionnaire wording.
What it does: Reverses the items loading negatively on the first principal component.
Effect on the result: Item statistics and alpha.
On the validation data: In bfi A1-A5 A1 is selected; in q1-q4 no item is selected.
Grouping variable
Concept: Compares the items across the levels of a grouping variable: summaries are given for each group, and the Kruskal-Wallis test checks whether the groups' response distributions differ in rank terms. The test suits ordinal data and does not assume normality.
Everyday example: The municipality splits the answers by district. If agreement with 'buses arrive on time' is 60% in one district and 25% in another, Kruskal-Wallis says whether this difference can be explained by chance.
What it does: Compares the items by a grouping variable (e.g. gender): n, mean, SD, median, top-2 percentage per group and the Kruskal-Wallis test.
Effect on the result: Group table only.
On the validation data: bfi gender, A1: H = 83.735, df 1. education 5 levels (rows with a missing group excluded); pisa CNT (text) 3 levels.
Additional output: Item normality
Concept: Summarises the shape of each item's response distribution with skewness and kurtosis, and its departure from normality with the Shapiro-Wilk test. Likert items are discrete and bounded, so they are not expected to be normal; these indicators are mainly useful for spotting ceiling or floor effects and answers piling up at one end.
Everyday example: If most passengers gave 4 or 5 on 'drivers are polite', the distribution is skewed to the left and there is a ceiling effect: the item can hardly tell satisfied passengers apart. This can guide rewording the item to discriminate better in the next survey.
What it does: Gives skewness and kurtosis (G1, G2) and their standard errors (Cramer, 1997; the exact formula depending on n), z values and the Shapiro-Wilk W (3 <= n <= 5000).
Effect on the result: Additional table only.
On the validation data: q1: SE(G1) 0.309, SE(G2) 0.608, W 0.9039, p 0.00018. In the pisa data n > 5000, so W is not computed.
Additional output: Diverging (neutral-centred) bar chart; Sort items in the chart by agreement
Concept: The diverging stacked bar chart places each item's disagreement responses to the left of a zero line and its agreement responses to the right, with neutral answers split across zero. Differences in agreement between items can thus be compared at a glance; the sort option orders the items by the difference between agreement and disagreement.
Everyday example: In the chart, the bar for 'drivers are polite' stretches to the right, while most of the bar for 'buses arrive on time' stays on the left. With the items sorted, the municipality sees the most problematic statements straight away at the bottom of the chart.
What it does: Draws a diverging stacked bar chart in the style of Heiberger and Robbins (2014) (disagreement on the left, agreement on the right, the neutral category split at zero); the sort option orders the items by (agree % - disagree %).
Effect on the result: Chart only; the numeric output does not change.
On the validation data: No numeric difference.
Save scale score (Sum score, Mean score)
Concept: Computes each respondent's scale score from the items after reverse coding and adds it to the dataset. The total score is computed only for people who answered every item, the mean score from the items answered; the mean score can be read on the original response scale (e.g. 1-5).
Everyday example: A 'public transport satisfaction score' is written next to each passenger. A passenger with a mean score of 2.1 is generally dissatisfied; the municipality can use this score in other analyses relating it to age or district.
What it does: Adds the sum (complete rows) or mean (available items) score after reverse coding to the dataset.
Effect on the result: New column only.
On the validation data: The q1-q4 and bfi pairwise mean scores are computed row by row.
Example on real data
Data: UCLA M255 survey data
Variables: items ITEM13-ITEM24 (12 five-point Likert items)
- Question: What is the response distribution of participants on the 12 Likert items (1-5) of the UCLA M255 survey, and does the item set form an internally consistent scale?
- Sample: 1365 of the 1428 participants answered all items (listwise deletion, 63 missing).
- Reliability: Cronbach's α = 0.912; the items are internally consistent at an "excellent" level.
- Descriptive: All item means are above the scale midpoint (3) (3.61-4.53). Agreement is highest for ITEM14 (mean 4.53, top-2 92.6%) and ITEM13 (4.46, 92.0%), lowest for ITEM24 (3.67, 57.7%) and ITEM20 (59.6%; 34.9% of the responses neutral).
- Distribution shape: The first items show strong left skew and peakedness (ITEM13 skewness -1.58, kurtosis 3.26); because of the ceiling effect, medians and percentages rather than means should be reported for these items.
- Limitation: Likert responses are on an ordinal scale; the mean and SD are only approximate summaries. Skewed items can violate the normality assumption of parametric tests.
- APA: Internal consistency of the 12-item scale (N = 1365) was high, α = .91; item means ranged from 3.61 to 4.53 and agreement (top-2) rates were 57.7%-92.6%.
Related concepts
Normality assumptionConfidence level and confidence intervalKurtosisReverse-coded itemsConfidence interval for a proportion (Wilson, Clopper-Pearson, Agresti-Coull, Jeffreys)Listwise and pairwise deletionFactor (component) loadingsShapiro-Wilk test
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.