Statistics Glossary → Exploratory Factor Analysis (EFA)
Exploratory Factor Analysis (EFA)analysis
A method that tries to explain the relationships among answers to many items by a few common factors that lie behind them and cannot be measured directly, showing which item belongs to which factor.
Concept
When the items in a survey are related to one another, it is usually because they reflect a common latent characteristic (job satisfaction, anxiety, self-efficacy and so on). Exploratory factor analysis (EFA) starts from the pattern of correlations among the items to reveal how many such latent characteristics there are and which one each item is related to. It is used in scale development to see whether the items fall into the intended dimensions and to provide evidence of construct validity.
According to the common factor model, each item consists of a weighted sum of the common factors plus an error specific to that item: x_j = λ_j1·F₁ + λ_j2·F₂ + … + e_j. The λ coefficients are the factor loadings and show how strongly the item is related to each factor; the part of an item's variance explained by the common factors is its communality (h²), the rest its unique variance. The analysis proceeds through three decisions: how many factors to extract (Kaiser criterion, parallel analysis, MAP), how to estimate them (maximum likelihood, principal axis factoring and so on), and how to rotate them so that the loading pattern becomes interpretable. Orthogonal rotations (varimax) keep the factors uncorrelated; oblique rotations (oblimin, promax) allow the factors to be correlated.
Unlike principal component analysis, EFA models only the variance shared among the items and keeps measurement error separate, which makes it more suitable for interpreting latent constructs. Before the analysis the suitability of the data is assessed with KMO and Bartlett's test, and a reasonably large sample (often a few hundred people) is recommended. The choice of the number of factors and of the rotation can change the result considerably, and naming the factors is the researcher's interpretation. EFA discovers a structure; testing whether a pre-specified structure fits the data requires confirmatory factor analysis.
When to use it
- When the aim is to discover, without prior knowledge, how many dimensions the items of a scale or survey fall into and which item belongs to which dimension, or to examine the structure of a newly developed scale for the first time.
- If a particular factor structure is known from theory or earlier studies and its fit to the data is to be tested, confirmatory factor analysis (a structural equation model) is used.
- If the aim is not to interpret latent constructs but to reduce many variables to fewer components while keeping the information, principal component analysis is more suitable.
An everyday example
A company gives its employees a survey of 12 statements about working life, such as 'I cannot finish my work within working hours', 'My manager listens to my ideas', 'I can't relax in the evenings for thinking about work' and 'My manager recognises my achievements'. By looking at the relationships among the answers, EFA may show that these 12 items fall into two common factors: the workload items have high loadings on one factor and the managerial support items on the other. Instead of 12 separate items, the company can then compare its departments with two scores called 'workload' and 'managerial support'.
Parameters in MerQur
Items
Concept: The items whose factor structure is sought; the analysis rests on their correlation matrix. The suitability of the data for factor analysis (KMO, Bartlett's test) is also computed from these items; adding an item unrelated to the scale blurs the structure.
Everyday example: The company selects the 12 statements about working life. If the survey's 'how many years have you worked here?' question were added to the items, it would load cleanly on no factor and make the structure harder to read.
What it does: Selects the items entering the analysis; KMO, Bartlett and the eigenvalues are computed from these items.
Effect on the result: All outputs.
On the validation data: M255 KMO 0.9344, Bartlett chi-square(66) = 8676.7115.
Number of factors (Automatic, Fixed 1-10, Parallel analysis, MAP)
Concept: Sets how many common factors are extracted, one of the most important decisions in EFA. The Kaiser criterion keeps factors with eigenvalues above 1 and often suggests too many; parallel analysis compares the observed eigenvalues with those of random data; Velicer's MAP method looks for the point at which the partial correlations remaining among the items, as factors are extracted, are smallest.
Everyday example: If the Kaiser criterion suggests three factors while parallel analysis and MAP suggest two, the third factor is most likely a chance leftover. If the company can interpret the two-factor solution meaningfully as 'workload' and 'managerial support', it prefers that solution.
What it does: Automatic = the number of factors with eigenvalue greater than 1 (Kaiser); Fixed 1-10 a number by hand; Parallel analysis (Horn; normal data, 100 replications, seed 42, mean criterion); MAP (Velicer, squared partial correlation).
Effect on the result: The number of extracted factors, and so all loadings and the explained variance.
On the validation data: M255 Kaiser 2 (eigenvalues 6.249 and 1.229), parallel 2, MAP 2. bfi Kaiser 6, parallel 5, MAP 5.
Extraction method (ML, PAF, Minres/ULS, GLS, PC)
Concept: The method by which the factor loadings are estimated. Maximum likelihood (ML) assumes multivariate normality and provides fit tests; principal axis factoring (PAF) and minres are less sensitive to distributional assumptions; principal components (PC) is based on the total variance rather than the common factor model.
Everyday example: If the employees' answers run from 1 to 5 and pile up at one end on many items, the normality assumption weakens, and PAF or minres gives a safer result. If the company wants to report the fit of the two-factor model with chi-square and RMSEA, it uses ML.
What it does: Selects the factor extraction method; the default is maximum likelihood. PAF and minres are common for Likert items; ML is used for fit tests.
Effect on the result: Loadings, communalities and fit indices.
On the validation data: M255, 3 factors, varimax; ITEM13 communality: PAF 0.6765, minres 0.6765, ML 0.6751, GLS 0.6705, PC 0.7398.
Rotation (Varimax, No rotation, Quartimax, Equamax, Oblimin, Promax, Geomin)
Concept: Rotation repositions the factor axes without changing the total variance explained or the communalities; the aim is a 'simple structure' in which each item loads strongly on one factor and weakly on the others. Varimax, quartimax and equamax keep the factors uncorrelated; oblimin, promax and geomin allow them to be correlated and also report the correlation between factors.
Everyday example: Before rotation most items load moderately on both factors and the factors are hard to name. After rotation the workload items stand out on one factor and the managerial support items on the other. If employees with heavy workloads are expected to perceive less support from their manager, oblimin, which allows the two factors to be correlated, is more realistic.
What it does: Rotates the factors with the GPA algorithm (tolerance 1e-8, at most 10000 iterations); a sign rule is applied, and the promax target is built from Kaiser-normalized varimax loadings (kappa = 4).
Effect on the result: The loading pattern, factor correlation and sums of squares; the communalities do not change.
On the validation data: M255 PAF 3 factors varimax sums of squares 2.950, 2.655, 1.415. Oblimin Phi12 = 0.610; geomin 0.6006; promax 0.6671.
Kaiser normalization (in rotation)
Concept: Before rotation, scales each item's loadings by its communality, so that items with high communality do not dominate the rotation and all items take part in the rotation criterion with equal weight. After rotation the loadings are converted back to their original scale.
Everyday example: If some workload items are very strongly related to the factors and some support items more weakly, a rotation without normalisation is shaped by the strong items. Normalisation gives every item an equal say in the rotation.
What it does: When on, normalizes the loadings by row (communality) before rotation (the default); when off, no normalization is done.
Effect on the result: The rotated loadings and sums of squares.
On the validation data: When off, PAF 3 varimax sums of squares 2.831, 2.345, 1.845.
Missing data (Listwise, Pairwise)
Concept: With listwise deletion, anyone who left even one item blank is removed from the analysis entirely. With the pairwise method each correlation is computed from everyone who answered those two items; more data are used, but because the correlations rest on different groups of people, the matrix may be inconsistent.
Everyday example: If 40 of 500 employees left at least one item blank, listwise deletion runs the factor analysis on 460 employees. The pairwise method also brings the items these 40 people did answer into the correlations.
What it does: Listwise drops rows with missing values; pairwise computes the correlations pair by pair.
Effect on the result: The correlation matrix and the Bartlett test.
On the validation data: Pairwise Bartlett chi-square 9140.86; the loadings are estimated from the pairwise correlation matrix.
Correlation type (Pearson, Polychoric)
Concept: The Pearson correlation treats items as continuous measurements. The polychoric correlation assumes a continuous, normally distributed tendency behind ordinal items with few categories and estimates the relationship between these tendencies; with few-category, skewed Likert items it can give more accurate loadings than Pearson.
Everyday example: Employees can express their feelings about workload only as a number from 1 to 5. The polychoric correlation estimates the relationship between the finer feelings behind these coarse numbers; especially when most answers gather at 4 and 5, Pearson can make the relationships look weaker than they are.
What it does: The polychoric option uses the two-step maximum likelihood polychoric correlation matrix for ordinal items with few categories; factor scores are computed on the Pearson scale.
Effect on the result: The correlation matrix and the loadings.
On the validation data: bfi 5 factors minres oblimin, with polychoric correlations: Bartlett chi-square 23261.83 (no continuity correction is applied to empty cells).
Additional output: Correlation matrix, significance, determinant, anti-image
Concept: The correlation matrix and its significance show the raw relationships among the items. A very small determinant means some items almost duplicate each other (multicollinearity). The MSA values on the diagonal of the anti-image matrix show how suitable each item is for factor analysis; items with low MSA are candidates for removal.
Everyday example: If 'I cannot finish my work within working hours' and 'My working hours are not enough' correlate at 0.95, the two repeat the same question and the determinant becomes very small. An item whose MSA is below 0.50 has too little in common with the others and can be dropped from the scale.
What it does: Gives the correlation matrix, one-tailed p values, the determinant (< 0.00001 indicates multicollinearity) and the anti-image matrices with MSA values.
Effect on the result: Additional table only.
On the validation data: M255 determinant 0.0016886; the anti-image matrices and MSA values are also given.
Additional output: Initial/extraction communalities and Total Variance Explained
Concept: The initial communality is a first estimate of how much each item is explained by all the other items (the squared multiple correlation); the extraction communality is the part of the item's variance explained by the extracted factors. The total variance explained table shows the variance accounted for by each factor and by all factors together.
Everyday example: If the item 'lighting in the office is adequate' has a communality of 0.10, the two factors explain only 10% of it; the item has nothing to do with workload or managerial support. If the two factors together explain, say, 55% of the items' variability, the structure summarises a substantial part of the relationships in the data.
What it does: Gives the initial communality (SMC; 1 for principal components), the communality after extraction and the sums-of-squares table.
Effect on the result: Additional table only.
On the validation data: ITEM13 SMC 0.5642.
Additional output: Goodness-of-fit test and indices (chi-square, TLI, RMSEA, BIC, RMSR)
Concept: With maximum likelihood extraction, assesses how well the model fits the data: the chi-square test checks whether the model reproduces the correlations exactly and, in large samples, finds even small departures significant; TLI and CFI close to 1, and RMSEA and RMSR (the size of the residual correlations) close to 0, indicate good fit; BIC helps compare different numbers of factors.
Everyday example: In data from 500 employees the chi-square test may be significant, yet if RMSEA is 0.05 and CFI 0.97, the two-factor model explains the relationships among the items well. If the three-factor model has a larger BIC, the extra factor is not worth the complexity.
What it does: Gives the Bartlett-corrected chi-square from the ML objective function, TLI, CFI, RMSEA (90% confidence interval), BIC and the residual RMSR.
Effect on the result: Fit table only.
On the validation data: M255 ML 3 factors: chi-square(33) = 135.353, TLI 0.976, CFI 0.988, RMSEA 0.0477 [0.0395, 0.0562], BIC -102.87.
Additional output: Sort loadings by size, suppress small ones, cross-loading warning; Loading suppression cut
Concept: Makes the loading table readable: it groups items by the factor they load most strongly on, hides loadings below a chosen threshold and lists items that load above the threshold on more than one factor (cross-loadings). Cross-loading items are examined closely in scale development because it is unclear which dimension they belong to. These options do not change the numeric result.
Everyday example: With loadings below 0.30 hidden, the workload items form a tidy block under the first factor and the support items under the second. If the item 'my manager balances my workload' loads 0.45 on both factors, it reflects both workload and support, and its wording may be revised.
What it does: Groups the items by the factor on which they load most; suppresses loadings below the cut (0.00, 0.30, 0.50); the cross-loading list threshold is >= .30. The cut is effective only when sorting is on.
Effect on the result: Display only; the numeric output does not change.
On the validation data: The numeric output does not change.
Factor scores (Regression, Bartlett, Anderson-Rubin, ten Berge) and Save factor scores to the dataset
Concept: A factor score is an estimate of each person's position on each factor. The regression method gives the scores most highly correlated with the factor; the Bartlett method produces unbiased scores; Anderson-Rubin scores are uncorrelated with one another and standardised; the ten Berge method preserves the correlations between factors. The save option adds the scores to the dataset as new columns.
Everyday example: A 'workload' score and a 'managerial support' score are written next to each employee. The company can compare departments with these scores, or examine in another analysis how the workload score relates to intention to leave.
What it does: Computes the factor score weights and row scores; the save option adds the scores as new columns such as FAC1_REG, FAC2_REG.
Effect on the result: Scores only.
On the validation data: The weights and row scores are computed with the regression, Bartlett and Anderson-Rubin methods (M255 PAF 3 varimax: ITEM13 F1 regression weight 0.4112).
Example on real data
Data: UCLA M255 survey data
Variables: items ITEM13-ITEM24 (12 attitude items)
- Question: How many latent factors underlie the 12 items of the UCLA M255 survey, and how do the items load on them?
- Assumption: KMO = 0.9344 (excellent) and Bartlett χ²(66) = 8676.71, p < .001; the data are suitable for factor analysis (listwise n = 1365).
- Number of factors: The Kaiser rule (eigenvalues 6.249 and 1.229 > 1), parallel analysis and Velicer's MAP all suggest 2 factors.
- Result: With maximum likelihood extraction and varimax rotation, F1 explains 28.8% of the variance and F2 26.3% (total 55.1%).
- Loading pattern: ITEM13-ITEM16 load highly on F1 (0.60-0.79), ITEM18-ITEM20 and ITEM22 on F2 (0.56-0.75). ITEM17, ITEM21, ITEM23 and ITEM24 load above 0.40 on both factors (cross-loading); these items should be reviewed in scale development.
- Limitation: There is a strong general factor (in PCA λ1 is 52% of the variance); an oblique rotation (oblimin, promax) may be more suitable than an orthogonal one. In a large sample the ML chi-square fit test is sensitive even to small deviations; the fit of the 2-factor solution should be judged together with RMSEA, CFI and TLI.
- APA: KMO = .93, χ²(66, N = 1365) = 8676.71, p < .001; the 2 factors extracted with maximum likelihood and varimax rotation explained 55.1% of the total variance.
Related concepts
Factor rotation (varimax, promax, oblimin)CommunalityFactor and component scoresFactor (component) loadingsParallel analysis (Horn) and the MAP criterionFactor extraction method (ML, PAF, minres)Polychoric and tetrachoric correlationCovariance and correlation matrix
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.