MerQuR+Statistics Glossary

Statistics Glossary → Weighted least squares (WLS)

Weighted least squares (WLS)concept

A regression estimation method that gives each observation its own weight and minimises the weighted sum of squared residuals; it is mainly used when observations have different variances.

Ordinary least squares (OLS) gives all observations equal importance and assumes a constant error variance. WLS minimises Σ wᵢ(yᵢ − ŷᵢ)². If the weights are chosen inversely proportional to the error variance (wᵢ = 1/σᵢ²), more precise observations contribute more and the estimates are the most efficient. When the observations are group means, the weight is usually the group size, because the variance of a mean is σ²/nᵢ.

If the weights are unknown they are estimated from the data, for example by assuming that the variance is proportional to a power of a predictor (wᵢ = 1/xᵢ^p). The uncertainty in the estimated weights is then not reflected in the standard errors, so combining WLS with robust (sandwich) standard errors is a safe approach. The 'weight' in WLS is not the same thing as a survey sampling weight, which shows how many people in the population an observation represents and requires design-based methods for correct standard errors.

An everyday example

A district studies the relationship between class mean scores and teacher experience in 30 schools. Some means are based on 10 pupils and others on 40. Because the mean of a 40-pupil class is far more precise, WLS gives each mean a weight proportional to its number of pupils, so the noisy means of small classes influence the result less.

In MerQur

Multiple linear regression has a 'WLS weight variable' (rows with a missing or ≤ 0 weight are excluded) and, under 'Model diagnostics', estimation of weights of the form w = 1/x^p over a power grid; nonlinear regression has a 'Weight (WLS)' option. GWR estimates by distance-weighted least squares at each point. In weighted linear regression the coefficients are computed by weighted least squares with the sampling weight, the 'Estimator' in path analysis includes a WLS ADF option, and in Variogram + Kriging the 'Fitting weights' option chooses the weight used in the weighted least-squares fit of the variogram model.

Analyses where this concept appears

Multiple Linear RegressionWeighted Linear Regression (Survey Regression)Nonlinear RegressionGWR (Geographically Weighted Regression)Path AnalysisVariogram + Kriging (Spatial Interpolation)

Related concepts

Ordinary least squares (OLS)Heteroscedasticity