MerQuR+Statistics Glossary

Statistics Glossary → Cronbach's Alpha and Reliability Analysis

Cronbach's Alpha and Reliability Analysisanalysis

An analysis that quantifies how consistently the items brought together to measure the same characteristic are answered, that is, the internal consistency reliability of a scale.

Concept

A scale tries to measure one characteristic (anxiety, satisfaction, an attitude and so on) with several items; the sum or mean of the items becomes the person's scale score. For this score to be meaningful, the items must measure the same thing, i.e. people must answer them consistently. Reliability analysis answers the questions 'do these items together form a consistent measure, and which item undermines that consistency?', and is a basic step in scale development and survey research.

Cronbach's alpha compares the sum of the individual item variances with the variance of the total score: α = k/(k − 1) · (1 − Σs²_i / s²_total), where k is the number of items. If the items are related, the variance of the total is much larger than the sum of the item variances and alpha approaches 1. The item-total correlation shows how well each item agrees with the sum of the others, and 'alpha if item deleted' how consistency would change if that item were removed. McDonald's omega uses the items' loadings on a common factor and is a more suitable reliability coefficient than alpha when the items do not contribute equally.

A commonly used benchmark for alpha is 0.70 or above, but this is a convention that varies with the context. Alpha rises automatically as the number of items grows; a high alpha does not prove that the scale is unidimensional, which should be examined with factor analysis. Alpha equals reliability under the assumption that all items measure the common characteristic equally strongly (tau-equivalence); when this is violated it can understate reliability. Negatively worded items must be reverse-coded before summing; otherwise alpha drops artificially.

When to use it

  • When the internal consistency of a scale or subscale made up of several items thought to measure the same characteristic must be reported, and problem items identified, before its total score is used.
  • If the items' loadings on the common factor differ clearly, McDonald's omega is used alongside alpha; if the number of dimensions of the scale is unknown, exploratory factor analysis comes first.
  • If the question is consistency of measurement over time, test-retest correlation or the intraclass correlation (ICC) is suitable; for agreement between raters, kappa or the ICC.

An everyday example

A hospital measures patient satisfaction with a six-item questionnaire: 'The nurses were attentive', 'The doctor answered my questions', 'My room was clean', 'The waiting time was far too long' and so on, each scored from 1 to 5. A satisfied patient gives high scores to the first three items but a low score to 'the waiting time was far too long'; this item is worded in the opposite direction and must be reversed before summing. If alpha comes out at 0.82 after reverse coding, the items are consistently measuring the same satisfaction. If alpha rises clearly when one item is deleted, that item may be measuring something different from the rest of the scale.

Parameters in MerQur

Items

Concept: The items making up the scale whose reliability is computed. Alpha rests on comparing the sum of these items' variances with the variance of the total score, so adding an item that does not belong to the scale, or leaving one out, directly changes the result.

Everyday example: The hospital selects the six satisfaction items. If the questionnaire's 'your age' question were added by mistake, alpha would drop because it has nothing to do with satisfaction, and the item-total correlation would reveal the error at once.

What it does: Selects the items forming the scale; alpha = k/(k-1) x (1 - sum s_i^2 / s_total^2).

Effect on the result: Alpha, standardized alpha, item-total correlation, alpha if item deleted and scale statistics.

On the validation data: q1-q4 alpha 0.8390, standardized alpha 0.8400. Fish1 alpha 0.8221, standardized alpha 0.9851.

Reverse-coded items

Concept: Reverses the scores of items worded in the opposite direction to the characteristic being measured, so that a high score means the same thing on every item. If a reversed item is summed without being recoded, it correlates negatively with the other items and lowers alpha markedly.

Everyday example: On the item 'the waiting time was far too long', a score of 5 means dissatisfaction. Reverse coding turns 5 into 1 and 4 into 2; now a high score on this item also means high satisfaction.

What it does: Reverses the marked items with x' = (min + max) - x; min and max are the extreme values observed across all items. Used with the theoretical key list for reverse-keyed items.

Effect on the result: Alpha, item-total r, alpha if deleted, scale mean and the saved score.

On the validation data: bfi A1-A5 uncorrected alpha 0.4306, 0.7038 when A1 is reversed. When q2 is reversed for testing, alpha drops from 0.839 to 0.0024.

→ Reverse-coded items

Missing data (Listwise (drop missing rows), Pairwise (per-calculation))

Concept: With listwise deletion, anyone who left even one item blank is removed entirely and all calculations use the same people. With the pairwise method, each covariance is computed from everyone who answered those two items; more data are used, but the calculations rest on slightly different groups of people.

Everyday example: If 30 of 400 patients left 'my room was clean' blank, listwise deletion computes alpha from 370 patients. The pairwise method also uses these 30 patients' answers to the other items.

What it does: Listwise uses complete rows; pairwise computes covariances pair by pair.

Effect on the result: Alpha, the confidence interval and the sample size.

On the validation data: Fish1 listwise n = 34, alpha 0.8221 [0.6994, 0.9029]; pairwise 0.8250 [0.7042, 0.9044]. M255 listwise n = 1365, alpha 0.9120; pairwise 0.9124 (smallest pairwise n = 1388).

→ Listwise and pairwise deletion

Confidence Level (%90, %95, %99, Other (custom))

Concept: Alpha too is an estimate computed from the sample; its confidence interval shows the range in which the true reliability may lie. The higher the confidence level, the wider the interval; with small samples it is especially wide.

Everyday example: If alpha is 0.82 with a 95% confidence interval of [0.79, 0.85], the scale's reliability is comfortably above the 0.70 benchmark. With only 40 patients the interval widens to something like [0.70, 0.90] and the result is less certain.

What it does: Sets the level of the Feldt (1965) confidence interval for alpha (and of the bootstrap interval).

Effect on the result: Only the interval bounds.

On the validation data: q1-q4: 90% [0.7751, 0.8888]; 95% [0.7604, 0.8965]; 99% [0.7291, 0.9104].

→ Confidence level and confidence interval

Automatic reverse coding (items loading negatively on the first component)

Concept: Infers from the data which items are reversed: it flips the items that load negatively on the first principal component, which summarises the common tendency of all items. This is for exploration only; which items are reversed should be known from the scale's theoretical key.

Everyday example: If the scoring key has been lost, automatic reverse coding notices that the 'waiting time' item runs opposite to the others and flips it. But if the scale is made up of two different dimensions, this method may flip the wrong item too, so the scale's manual remains the main reference.

What it does: Reverses the items loading negatively on the first principal component. For exploration only; in a multidimensional scale it should not replace the theoretical key.

Effect on the result: Alpha and all related item statistics.

On the validation data: In bfi A1-A5 only A1 is selected, alpha 0.7038. In the 25 items A1, C4, C5, E1, E2, N1-N5, O2, O4, O5 are selected, alpha 0.8246.

→ Reverse-coded items

ω_h number of group factors

Concept: McDonald's omega is a reliability coefficient computed from the items' loadings on a common factor and is more suitable than alpha when the items contribute with different strength. When group factors are specified, omega hierarchical (ω_h) shows how much of the total-score variance comes from a single general factor.

Everyday example: If patient satisfaction has two subdimensions, such as 'staff' and 'environment', omega hierarchical says how far the total satisfaction score reflects a single general sense of satisfaction. If ω_h is low, the total score behaves like a mixture of two separate subdimensions.

What it does: McDonald's omega_t (one factor, minres) is always computed; with m >= 2 group factors and k >= 2m items the Schmid-Leiman omega_h and omega_t are also computed.

Effect on the result: Only the omega outputs; items with a negative g loading are reversed automatically and listed, and a warning is given if the keying differs from alpha.

On the validation data: M255 m = 3: omega_h 0.7839, omega_t 0.9331; m = 4: 0.7905 and 0.9363. In q1-q4 k < 6, so a note says that ω_h cannot be computed. The one-factor omega_t in q1-q4 is 0.8416.

→ McDonald's ω (omega) reliability coefficient

Additional output: Inter-item correlation/covariance matrix + summary

Concept: The inter-item correlation and covariance matrices show the raw information on which alpha rests. The mean inter-item correlation is a measure of consistency that does not depend on the number of items; a very low value can indicate that the items have little in common, a very high one that they repeat each other.

Everyday example: If the correlation between 'the nurses were attentive' and 'the nurses were polite' is 0.90, these two items are asking almost the same question twice. If 'my room was clean' correlates around 0.10 with the others, this item is looking at a separate topic from the rest of the scale.

What it does: Gives the item covariance and correlation matrices and summary statistics (mean, minimum, maximum, range, ratio, variance).

Effect on the result: Additional table only.

On the validation data: The q1-q4 covariance matrix starts with the values 1.168, .557 ...

→ Covariance and correlation matrix

Additional output: Other reliability models: split-half + Guttman λ1-λ6

Concept: Split-half reliability divides the scale into two halves and adjusts the correlation between the two halves' scores to the full-length scale with the Spearman-Brown formula: r_full = 2r / (1 + r). The Guttman lambda coefficients are lower bounds for reliability based on different assumptions; λ3 equals Cronbach's alpha.

Everyday example: The first three and the last three of the six items are summed separately and the two halves' scores are compared. If the correlation between the halves is 0.70, the Spearman-Brown correction estimates the reliability of the six-item scale at about 0.82.

What it does: Splits the item set into the first ⌈k/2⌉ items and the rest; computes the Spearman-Brown and Guttman split-half values and Guttman's lambda 1-6.

Effect on the result: Additional table only.

On the validation data: q1-q4 part alphas 0.6763 and 0.7410, between-forms r = 0.7406, Spearman-Brown 0.8510, Guttman split-half 0.8492; lambda 1-6 = 0.6293, 0.8401, 0.8390, 0.8492, 0.8246, 0.8011.

→ Split-half reliability and Guttman's λ

Split-half: items in part 1

Concept: Sets the point at which the scale is split in two. Split-half reliability depends on which items fall into which half; different split points can give different results, so a single split-half value should be interpreted with care.

Everyday example: If the first three items concern staff and the last three the environment, a first-three/last-three split compares two different topics and makes reliability look low. If the split point is changed so that each half gets items from both topics, the result may change.

What it does: The user sets the split point (0 = automatic, the first ⌈k/2⌉ items). k1 is the number of items in the first part; the second part has k - k1 items.

Effect on the result: The part alphas, between-forms r, Spearman-Brown equal/unequal length and Guttman split-half values.

On the validation data: attitude (k = 7) with k1 = 2 (second part 5 items): r = 0.6243789807, Spearman-Brown 0.7687602316, unequal length 0.7970726965, Guttman 0.7171060704.

→ Split-half reliability and Guttman's λ

Additional output: Scale ANOVA + Tukey additivity + Friedman/Cochran + Hotelling T²

Concept: The scale ANOVA splits the total variability into persons, items and residual, and tests whether the item means differ. Tukey's test of additivity checks whether person and item effects are additive; the Friedman or Cochran test compares the item means non-parametrically, and Hotelling's T² does so multivariately.

Everyday example: If patients give 'the doctor answered my questions' an average of 4.5 and 'my room was clean' 3.2, the items are worded at different levels of difficulty or satisfaction. These tests show whether such differences between item means can be explained by chance.

What it does: Gives the persons/items/residual sums of squares, Tukey's 1-df test for nonadditivity and the power transformation estimate, Friedman's chi-square and Hotelling's T2.

Effect on the result: Additional table only.

On the validation data: q1-q4 Tukey F = 0.068, p = 0.794, power 0.599. Friedman chi-square 4.923, p = 0.178 (with within-subject ranks). Hotelling T2 = 6.207, F(3, 57) = 1.999, p = 0.124.

→ Analysis of variance (ANOVA) table

Additional output: Bootstrap CI for α (1000 resamples)

Concept: Computes the confidence interval of alpha by looking at how alpha is distributed over thousands of resamples drawn from the respondents with replacement. It does not need the distributional assumptions of the formula-based interval.

Everyday example: The 400 patients' answer sheets go into a bag, 400 sheets are drawn with replacement and alpha is computed; this is repeated 1000 times. The range covered by the middle 95% of these alphas gives the confidence interval.

What it does: Computes a percentile bootstrap interval for alpha (B = 1000, seed 42); does not assume normality.

Effect on the result: Interval only.

On the validation data: q1-q4 [0.7510, 0.8930]; the bootstrap interval carries Monte Carlo error.

→ Bootstrap (resampling)

Additional output: Ordinal α (polychoric correlations)

Concept: Likert items with few categories can be treated as coarse ordinal measurements of an underlying continuous tendency. Ordinal alpha uses the polychoric correlation, which estimates the relationship between these continuous tendencies, instead of the Pearson correlation; it corrects the tendency of Pearson-based alpha to understate reliability as the number of categories falls.

Everyday example: A patient can express their view on 'cleanliness' only as a number from 1 to 5, whereas their actual view lies on a finer scale. Ordinal alpha tries to estimate how consistent the fine-grained views behind these coarse scores are.

What it does: Gives alpha from the polychoric correlation matrix for Likert items with few categories (Zumbo et al., 2007); a two-step polychoric estimate is used.

Effect on the result: Additional output only.

On the validation data: q1-q4 0.8695; bfi A (A1 reversed) 0.7596. The polychoric correlations are estimated with the two-step method (thresholds first, then the correlation).

→ Polychoric and tetrachoric correlation

Additional output: Parallel / strictly parallel model

Concept: The parallel model assumes that all items measure the true score with equal strength and have equal error variance; the strictly parallel model also requires equal item means. The fit of these models tests whether assumptions stricter than those underlying alpha hold in the data.

Everyday example: The parallel model assumes the six items measure satisfaction like 'copies of one another'. If the chi-square test rejects this, some items reflect satisfaction more strongly than others; more flexible coefficients such as omega are then more suitable.

What it does: Fits the parallel (equal loadings, equal residual variances) and strictly parallel (also equal intercepts) models by normal-theory maximum likelihood; gives the common variance, true and error variance, reliability, chi-square and df.

Effect on the result: Additional table only.

On the validation data: attitude parallel chi-square 14.9135359033 (df 8, p = 0.0608486662), reliability 0.8514004611. Strictly parallel chi-square 61.9029630389 (df 11).

→ Parallel and strictly parallel measures model

Subscales

Concept: If a scale consists of several subdimensions, the reliability of each should be computed separately; if subscale scores are to be used on their own, each must be internally consistent.

Everyday example: The hospital splits the items into two subdimensions, 'Staff: nurse, doctor, information; Environment: cleanliness, quietness, waiting'. If alpha is 0.85 for staff and 0.62 for environment, the environment score should be interpreted with more caution.

What it does: Computes a separate alpha for subscales given in the form "Name: item1, item2; Name2: ...".

Effect on the result: Subscale table only.

On the validation data: bfi (keyed, n = 2436) A 0.7158, C 0.7373, E 0.7651, N 0.8169, O 0.6078.

Save scale score (Do not save, Sum score, Mean score)

Concept: Computes each person's scale score from the items after reverse coding and adds it to the dataset. The total score is computed only for people who answered every item; the mean score is computed from the items answered, so it also gives a value for people with a few missing items and can be read on the original response scale (e.g. 1-5).

Everyday example: A 'satisfaction score' is written next to each patient. A patient with a mean score of 4.2 lies between 'satisfied' and 'very satisfied' on the 1-5 scale; this score can later be used to compare wards.

What it does: Adds the sum (complete rows) or mean (available items) score of the items after reverse coding to the dataset as a new column.

Effect on the result: New column only.

On the validation data: The scores are computed row by row; the bfi pairwise mean score covers 2800 rows.

Example on real data

Data: Revelle, R psych bfi

Variables: items A1-A5 (agreeableness subscale; A1 reverse-keyed)

  • Question: Do the five items (A1-A5) of the bfi Agreeableness subscale form an internally consistent scale when summed without correcting the reverse item?
  • Result: Cronbach's α = 0.4306, 95% CI [0.3960, 0.4638] (n = 2709, 91 rows with missing values dropped); far below the 0.70 criterion, "unacceptable" internal consistency.
  • Item analysis: The corrected item-total correlation of A1 is negative (-0.3114) and α rises to 0.7180 if A1 is deleted; the item-total r of the other items is 0.37-0.48.
  • Reason: A1 ("I am indifferent to the feelings of others") is a reverse-keyed item; summing it without reverse coding lowers alpha. With A1 reversed, α = 0.7038.
  • Additional finding: Standardized α 0.4574, mean inter-item r 0.1443; McDonald's ω_t = 0.7237, but since A1 is reversed automatically in the ω calculation, it does not use the same keying as α.
  • Limitation: Alpha assumes unidimensionality and equal loadings (tau-equivalence); reverse items should first be corrected with the key list.
  • APA: α = .43 (5 items, N = 2709); McDonald's ω = .72.

Related concepts

Analysis of variance (ANOVA) tableSplit-half reliability and Guttman's λReverse-coded itemsMcDonald's ω (omega) reliability coefficientParallel and strictly parallel measures modelPolychoric and tetrachoric correlationCovariance and correlation matrixFactor (component) loadings

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.