MerQuR+Statistics Glossary

Statistics Glossary → Single imputation (mean, median, kNN filling)

Single imputation (mean, median, kNN filling)concept

Completing the data by replacing each missing value with a single estimated value (the column mean or median, or the average of similar observations).

With mean or median imputation, every missing cell of a variable receives the same value. With k-nearest-neighbour (kNN) imputation, the k observations most similar on the other variables to the one with the missing value are found, and the average of their values is used; this partly preserves the relationships between variables. Single imputation is a practical way to run methods that need complete data, such as clustering or dimension reduction, without dropping incomplete rows.

The price is that uncertainty is ignored: imputed values are treated as if they were real observations. Mean imputation shrinks the variable's variance, pulls correlations towards zero and makes standard errors too small, and the distortion grows with the share of missing data. kNN is less distorting but is sensitive to scaling and to the choice of k. When the aim is inference (p values, confidence intervals), methods that account for uncertainty, such as multiple imputation or FIML, should be preferred.

An everyday example

In a class of 30, height is missing for 5 pupils and the class mean is 160 cm. Mean imputation writes 160 cm for all five, artificially narrowing the class's height distribution. kNN imputation instead assigns each of them the average height of the 5 pupils most similar in age and weight.

In MerQur

The 'Missing data' option in K-Means and Hierarchical Clustering, DBSCAN / HDBSCAN, Principal Component Analysis, t-SNE and UMAP (drop, fill with mean, median or KNN; filling is done on the raw scale, before transformation); 'Missing data' in PLS-SEM (mean replacement is the default).

Analyses where this concept appears

K-Means ClusteringHierarchical ClusteringDBSCAN / HDBSCANPrincipal Component Analysis (PCA)t-SNE (t-Distributed Stochastic Neighbor Embedding)UMAP (Uniform Manifold Approximation and Projection)PLS-SEM (Partial Least Squares Path Modeling)

Related concepts

Listwise and pairwise deletionMultiple imputation by chained equations (FCS/MICE) and PMM