Statistics Glossary → Weighted Linear Regression (Survey Regression)
Weighted Linear Regression (Survey Regression)analysis
A linear regression that relates a continuous outcome to predictors in survey data collected under a complex sampling design, estimating the coefficients with sampling weights and the standard errors in a design-correct way.
Concept
When survey data are used to ask questions such as 'how are sleep and exam performance related in the population?', ordinary regression runs into two problems. First, if the sample is not a proportional miniature of the population, the coefficients may reflect the sample's peculiar make-up rather than the population. Second, people selected from the same school or neighbourhood resemble one another, so the observations are not independent and ordinary standard errors are usually too small, producing more 'significant' results than warranted.
Design-based regression estimates the coefficients by weighted least squares: each observation is weighted by the share of the population it represents. The coefficient is then an estimate of the regression line that would be obtained if the whole population were measured. Standard errors are computed from the stratum and cluster structure (by Taylor linearisation or replication methods), and tests use design degrees of freedom, based on the numbers of clusters and strata rather than respondents. The joint significance of predictors is tested with a Wald F test.
Coefficients are read as in ordinary regression: the average change in the outcome for a one-unit increase in a predictor, holding the others fixed. The linearity assumption still applies, but assumptions such as normal, equal-variance errors matter less for design-based inference. A marked difference between weighted and unweighted coefficients can signal that the model works differently in some groups (e.g. a missing interaction). The results of this analysis are descriptive too; causation can only be claimed with support from design and theory.
When to use it
- When the relation between a continuous outcome and predictors is to be estimated at population level from a national or regional survey with weights, strata and clusters.
- If the data come from a simple random sample, use ordinary multiple linear regression; if the outcome is binary, use weighted logistic regression.
- If the aim is not a single population-level coefficient but modelling the variation between clusters such as schools or neighbourhoods itself, multilevel (mixed-effects) models are more suitable.
An everyday example
In the province-wide middle-school survey, 2000 pupils' maths scores, daily screen time and sleep time are recorded. Weighted linear regression might find that, holding sleep fixed, each extra hour of screen time is associated with a 2.5-point lower maths score on average. Because the similarity of pupils in the same school is taken into account, this coefficient's standard error is larger than in ordinary regression; a weak effect that looks 'significant' in an ordinary analysis may lose its significance once the design is specified correctly.
Parameters in MerQur
Predictors (X) and Target (Y)
Concept: The dependent variable is the continuous outcome to be explained; the predictors are the variables thought to explain it. A categorical predictor enters the model as dummy (0/1) variables, each comparing a level with a reference level.
Everyday example: Maths score is the dependent variable; screen time, sleep time and breakfast habit (yes/no) are the predictors. The breakfast coefficient says how many points higher, on average, pupils who eat breakfast score with everything else equal.
What it does: Selects the dependent variable and the predictors; dummy variables are built for categorical predictors. If a strata, cluster or FPC column is also selected as a predictor, the model is built as the user requested.
Effect on the result: Coefficients, SE, t, p and Wald F tests.
On the validation data: IceCream, Spending ~ Income + Kids: unweighted (SRS) Income coefficient 0.77533.
Weight (required)
Concept: The sampling weight shows how many people in the population each observation represents. In regression, weights make the coefficients reflect the composition of the population rather than of the sample; multiplying all weights by the same constant leaves the coefficients unchanged.
Everyday example: If rural schools were oversampled and the effect of screen time on maths is weaker in rural areas, an unweighted regression understates the province-wide relation. The weights give urban pupils influence in line with their share of the population.
What it does: Sampling weight (inverse of the selection probability). Coefficients are computed by weighted least squares and SE by Taylor linearization.
Effect on the result: Coefficients and SE; the weight scale does not change the coefficient.
On the validation data: IceCream Income coefficient 0.77533 (the same in all four sources).
Strata
Concept: Stratification divides the population into non-overlapping groups and samples each separately. In regression, stratum information affects not the coefficients but their standard errors and the degrees of freedom of the tests.
Everyday example: Once the urban and rural strata are declared, the screen-time coefficient stays the same, but its uncertainty is computed in line with the design.
What it does: Computes the variance from the deviations of within-stratum PSU totals; design degrees of freedom = PSU - strata.
Effect on the result: SE, confidence interval and degrees of freedom; coefficients do not change.
On the validation data: apistrat ell coefficient SE: unstratified 0.291733, stratified by school type 0.2883.
Cluster / PSU
Concept: The cluster (PSU) is the unit within which people are selected as a group. Because the residuals (the part the model fails to explain) of people in the same cluster are alike, effective information falls and the coefficients' standard errors usually grow.
Everyday example: If a school has excellent teachers, all its pupils score higher than the model predicts. This shared deviation shows that its 40 pupils do not carry 40 independent pieces of information; declaring the school as the cluster lets the standard error account for this.
What it does: Defines the primary sampling unit; because of within-cluster similarity, the SE generally increases.
Effect on the result: SE and degrees of freedom (PSU - strata).
On the validation data: apiclus1 (clustered by dnum) SE 0.28088, df 14.
FPC (population)
Concept: The finite population correction shrinks the variance by multiplying it by (1 − n/N) when a substantial part of the population is sampled. With a small sampling fraction its effect is negligible.
Everyday example: If half the rural schools entered the sample, the uncertainty coming from that stratum is reduced accordingly and the coefficients' standard errors shrink a little.
What it does: Finite population correction; the variance is multiplied by (1 - n_h/N_h).
Effect on the result: SE and interval become smaller.
On the validation data: apistrat without FPC SE 0.2883, with FPC 0.283947.
Single-PSU stratum (No contribution (certainty, default), Centered at the average (adjust))
Concept: If a stratum has only one cluster, the between-cluster variation within it cannot be computed. The option decides whether such a stratum contributes nothing to the variance or whether a more cautious contribution is computed relative to the average of all clusters.
Everyday example: If one district stratum had only a single school sampled, how much the relation varies from school to school there cannot be known; the cautious option reflects this unknown in the standard errors.
What it does: Determines the variance contribution of a stratum with a single PSU (no contribution, or centered at the average).
Effect on the result: SE only, when a single-PSU stratum is present.
On the validation data: apiclus1 no contribution SE 0.190768; centered at the average 0.190778.
Also add the stratum to the model as a fixed effect (dummies)
Concept: Adding the stratum to the model as a fixed effect gives each stratum its own intercept. The coefficients then show the within-stratum relation, purged of differences between strata; otherwise the stratum is used only in computing the standard errors.
Everyday example: If urban pupils have both more screen time and higher maths scores, the relation found without the stratum partly reflects the urban-rural difference. With the stratum added, the question becomes 'how is screen time related to scores among pupils in the same stratum?'.
What it does: Adds the stratum to the model as dummy fixed effects (coefficients become within-stratum effects); the default is off (the stratum is only in the variance design; Lumley, 2010).
Effect on the result: Coefficients, SE and tests; the stratum dummies also enter the overall model test. Enabled only when a stratum is selected.
On the validation data: apistrat meals coefficient: off -3.14154, on -3.40419 (the stratum dummies _str_H and _str_M are added to the model). The coefficients are the same as in R and SPSS.
Confidence Level (%90, %95, %99, Other (custom))
Concept: Sets how much assurance the coefficients' confidence intervals give of covering the true population coefficient. The intervals use a t distribution with design degrees of freedom.
Everyday example: If the screen-time coefficient is −2.5, you can say with 95% confidence that the true effect lies between, say, −4.0 and −1.0 points; asking for 99% widens the interval.
What it does: Sets the level of the coefficient confidence intervals (t distribution).
Effect on the result: Intervals only.
On the validation data: apistrat meals 95% CI [-3.710, -2.573], built with t(194).
Reference levels
Concept: The coefficients of a categorical predictor show differences relative to the chosen reference (baseline) level. Changing the reference does not change the model itself or its fit, only which comparisons can be read directly.
Everyday example: If mother's education is coded 'primary', 'secondary' and 'university', choosing 'primary' as the reference makes the coefficients show directly how many points ahead the secondary and university groups are of the primary group.
What it does: Sets the comparison level of a categorical predictor ('column=level; column2=level'); if empty, the first level alphabetically.
Effect on the result: Dummy coefficients are re-expressed; model fit and the term test do not change.
On the validation data: apistrat api00 ~ stype: with base level E the H coefficient is -93.2239; with base level M the E coefficient is 55.7732.
Interaction terms
Concept: An interaction term tests whether the effect of one predictor changes with the level of another. If there is an interaction, the main effects cannot be interpreted on their own, because the size of the effect depends on the other variable.
Everyday example: The negative effect of screen time on maths may be stronger in pupils who sleep little. The 'screen time × sleep time' term tests this; if significant, the effect of screen time should be described separately for each level of sleep.
What it does: Adds product terms in the form 'a*b' (for categorical variables, all dummy products; the main effects must be selected separately).
Effect on the result: Additional coefficients and the term test.
On the validation data: apistrat ell*meals coefficient 0.0102128; stype*meals term F = 0.49598.
Subpopulation (domain) column and Domain value
Concept: When the model is fitted for a subpopulation only, the other observations are given zero weight rather than deleted, so the full design (all clusters and strata) is kept for computing standard errors. Filtering the data beforehand produces wrong standard errors.
Everyday example: To fit the model for girls only, 'sex = girl' is given as the subpopulation. Boys stay in the dataset with zero weight, so even schools where no girls happened to be sampled are not lost from the design information.
What it does: Fits the model only for the subpopulation with the given value; other observations receive zero weight and the design is fully preserved (filtering the data gives a wrong SE).
Effect on the result: Coefficients come from the subpopulation; design information comes from the full design.
On the validation data: In the apistrat comp_imp = Yes subpopulation, the meals coefficient is -3.39153; the design degrees of freedom is 197.
Additional output: Standardized coefficients (β*)
Concept: A standardised coefficient is the coefficient obtained when the predictor and outcome are expressed in standard-deviation units. It lets you compare the effects of predictors measured in different units on the same scale.
Everyday example: If screen time is measured in hours and sleep in minutes, the raw coefficients cannot be compared directly. If the standardised coefficients are −0.25 for screen time and 0.15 for sleep, screen time has the larger effect per standard-deviation change.
What it does: Computes beta* = beta x s_w(x) / s_w(y) (weighted standard deviations); provides a scale-free comparison.
Effect on the result: An additional table only.
On the validation data: For apistrat meals, beta* = -0.752002.
Additional output: Save predictions and residuals to the dataset
Concept: Adds to the dataset the value the model predicts for each observation and the difference between the observed and predicted values (the residual). Residuals are used to see where the model works poorly and to check its assumptions.
Everyday example: If a pupil's predicted maths score is 65 and their actual score 85, the residual is +20. Large positive residuals clustering in the same schools may show that the model misses a factor specific to those schools.
What it does: Adds SVY_PRED (prediction) and SVY_RESID (residual) columns to the dataset.
Effect on the result: New columns only.
On the validation data: Row 1 prediction 707.685, residual 132.315 (the three programs the same).
Custom contrast (L)
Concept: A contrast tests a linear combination of coefficients (e.g. the difference or sum of two coefficients) as a single hypothesis. It enables comparisons that do not appear directly in the model.
Everyday example: If mother's education has 'primary' as its reference, the model compares secondary and university only with primary. A 'university − secondary' contrast tests directly whether those two groups' scores differ.
What it does: Tests linear combinations of the coefficients (e.g., the difference between categorical levels); for each row it gives the estimate, SE, t, p and confidence interval, with a joint F test. The joint F and the row t tests use ν + 1 - k denominator degrees of freedom.
Effect on the result: The contrast table only.
On the validation data: apistrat, stype: the M - H contrast is given with estimate, SE, t, p, confidence interval and joint F under Taylor and JKn variance; degrees of freedom 193.
Variance method (Taylor (default), Jackknife (JK1 / JKn), BRR (balanced half-samples), Fay BRR, Bootstrap (Rao-Wu), Supplied replicate weights) and related fields (Fay coefficient (rho), Bootstrap replicates, Random seed, Replicate weight columns, Replicate type (JK1, JKn, BRR, Fay, bootstrap, other), Scale, Replicate scales (rscales), Deviations from)
Concept: The standard error can be computed by formula-based Taylor linearisation or by replication methods (jackknife, balanced repeated replication, bootstrap). In replication methods the model is refitted in each design-based subsample, and the spread of the coefficients across these fits gives the standard error.
Everyday example: In the jackknife, schools are left out one at a time and the regression is refitted. If the screen-time coefficient stays around −2.5 whichever school is removed, the result is stable and the standard error is small.
What it does: Computes the standard error by resampling instead of Taylor; the model is re-estimated in each replicate and the Wald tests are based on the replicate covariance. The design degrees of freedom is the rank of the replicate weights - 1.
Effect on the result: SE, t, p, interval and Wald F; coefficient estimates do not change.
On the validation data: For all replication methods, coefficient, SE, t, p, interval and Wald F are given. Example: apistrat JKn Wald F 130.674650601, denominator df 194.
Example on real data
Data: California API apistrat, R survey package (Lumley)
Variables: target api00, predictors ell (English learners %), meals (free meals %), mobility (new students %); weight pw, stratum stype
- Question: In a stratified school sample (weight pw, stratum stype), do the share of English learners (ell), the share receiving free meals (meals) and student mobility (mobility) predict the API score (api00)?
- Model: Design-based weighted linear regression, Taylor SE; coefficient t tests with t(ν + 1 - k) = t(194); n = 200, R² = .66.
- Overall test: Wald F(3, 194) = 132.78, p < .001; adjusted Wald F(3, 195) = 131.43, p < .001. The model as a whole is significant.
- Coefficients: meals b = -3.14 (SE = 0.29), t = -10.90, p < .001, 95% CI [-3.710, -2.573]: a 1-point increase in the free-meal share is associated with a drop of about 3.1 points in API. ell (b = -0.48, p = .228) and mobility (b = 0.23, p = .576) are not significant.
- Limitation: Deviance and AIC are computed with raw weights; compare them only between models with the same weights.
- APA: Overall model (adjusted Wald): F(3, 195) = 131.43, p < .001; meals: b = -3.14, SE = 0.29, t(194) = -10.90, p < .001.
Related concepts
Primary sampling unit (PSU) and cluster samplingStratified sampling (design stratum)Finite population correction (FPC)Taylor linearization and replication variance methodsStratified analysis (stratum variable)Standardized regression coefficient (β)Sampling weight (design weight)Domain (subpopulation) estimation
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.