MerQuR+Statistics Glossary

Statistics Glossary → Eigenvalue and eigenvector

Eigenvalue and eigenvectorconcept

The vectors that a square matrix (e.g. a correlation or covariance matrix) only rescales without changing their direction (eigenvectors), and the rescaling factors themselves (eigenvalues).

For a matrix A, a vector v satisfying A·v = λ·v is an eigenvector and the number λ is its eigenvalue. In statistics the eigenvalues met most often are those of correlation or covariance matrices. In principal component analysis, each eigenvector gives the direction of a component (the weights of the variables) and the corresponding eigenvalue the variance that component explains. For a correlation matrix the eigenvalues add up to the number of variables, so dividing an eigenvalue by the number of variables gives the proportion of variance explained by that component.

Eigenvalues guide the decision on how many factors to retain in factor analysis: the Kaiser criterion keeps eigenvalues greater than 1 (and usually suggests too many factors), the scree plot looks for an elbow in the eigenvalues, and parallel analysis compares the observed eigenvalues with those from random data and is regarded as the most reliable method. Eigenvalues very close to zero indicate strong multicollinearity among the variables, which is why the condition index in regression is computed from them. Eigenvalues are also key statistics in canonical correlation, correspondence analysis and the Johansen cointegration test.

An everyday example

If a student's maths, physics and chemistry marks are closely linked, the three marks are largely moving along a single shared direction of 'numerical ability'. If the first eigenvalue of their correlation matrix comes out at, say, 2.4, then 80% of the three variables' total variance (3) is explained by that one direction, and the eigenvector shows how much weight that direction places on each subject.

In MerQur

It is used through the 'Eigenvalues' extra output in Canonical Correlation and through the 'MAXEIGEN (divisive)' threshold in Variable Clustering. It also appears in the output of Principal Component and Exploratory Factor Analysis, Correspondence Analysis, Multiple Linear Regression (multicollinearity diagnostics), Item Response Theory, SAR, Cointegration and Response Surface Regression.

Analyses where this concept appears

Canonical CorrelationCorrespondence Analysis (CA / MCA)Variable Clustering (VARCLUS)Multiple Linear RegressionPrincipal Component Analysis (PCA)Exploratory Factor Analysis (EFA)Item Response Theory (IRT / Rasch)SAR (Spatial Autoregressive Lag)Cointegration (Johansen / Engle-Granger)Response Surface Regression (RSREG)

Related concepts

Kaiser criterion (eigenvalue > 1)Scree plotProportion of explained variance