Statistics Glossary → Weighted Mean (Survey Means)
Weighted Mean (Survey Means)analysis
An analysis that estimates the population mean of a variable, together with a design-correct standard error, from survey data collected under a complex sampling design (weights, strata, clusters).
Concept
Large surveys are rarely based on simple random sampling. To keep costs down, schools or neighbourhoods (clusters) are selected first, regions are split into strata, and some groups are deliberately oversampled so that small groups are represented. Each respondent then has a different probability of selection, so the plain arithmetic mean of the data misrepresents the population, and a standard error computed with the ordinary formula is usually too small.
The weighted mean multiplies each observation by its sampling weight, the inverse of its selection probability: ȳ_w = Σ wᵢyᵢ / Σ wᵢ. A respondent with a weight of 200 stands for 200 people like them in the population. The standard error is computed from the design itself: stratification often improves precision because variance then comes only from differences within strata, while clustering reduces the effective amount of information, since people in the same cluster resemble one another, and inflates the standard error. The design effect (DEFF) is the ratio of the variance under the design to the variance of a simple random sample of the same size.
A correct result requires the weight, stratum and cluster information to be in the data file and to enter the analysis together. A DEFF above 1 means the design reduces precision, below 1 that it improves it. Subgroup (domain) estimates should be produced by keeping the full design rather than filtering the data, otherwise the standard errors are wrong.
When to use it
- When estimating the population mean of a variable from data with weights, strata and clusters, such as national health, education, household or labour-force surveys.
- If the data come from a simple random sample with no weights, ordinary descriptive statistics and a one-sample t-test are enough.
- For the proportions of a categorical variable use the weighted frequency, for population totals the weighted total, and for relations between variables weighted linear or logistic regression.
An everyday example
To find out how much screen time middle-school pupils have each day, a province splits its schools into urban and rural strata, selects schools within each, and surveys the pupils in the selected schools. Because rural schools are few, relatively more of them were sampled, so rural pupils carry smaller weights. The plain mean might be 3.4 hours while the weighted mean is 3.7 hours, because pupils in the crowded urban schools make up a larger share of the population. And because pupils in the same school have similar habits, the standard error is larger than it would be if 2000 pupils had been chosen by simple random sampling.
Parameters in MerQur
Weight (required)
Concept: The sampling weight shows how many people in the population a respondent stands for; it is essentially the inverse of the selection probability, possibly further adjusted for non-response or to match known population distributions. Weights reduce the influence of oversampled groups and increase that of undersampled ones.
Everyday example: If a pupil in a rural school had a 1 in 50 chance of selection and a pupil in an urban school a 1 in 200 chance, their weights are 50 and 200. In the weighted mean the urban pupil's screen time counts four times as much, because there are four times as many pupils like them in the population.
What it does: The sampling weight w_i (inverse of the selection probability; the final weight with nonresponse or calibration adjustment if any). Point estimates are weighted; the SE is computed from weighted score totals by Taylor linearization.
Effect on the result: The estimate and SE; the weight scale does not change the mean, but changes the total and the (without-replacement) DEFF denominator.
On the validation data: IceCream SRS (total 4000): unweighted mean 8.75, SE 0.845139; with stratified weights mean 9.1413, SE 0.865628.
Stratum
Concept: Stratification divides the population into predefined, non-overlapping groups and draws a separate sample from each. Because differences between strata do not contribute to sampling uncertainty, the standard error shrinks when the strata differ markedly from one another.
Everyday example: If urban pupils have clearly more screen time than rural pupils, making town and countryside separate strata guarantees that both are represented in the right proportion and makes the provincial mean more precise.
What it does: Computes the variance from the deviations of PSU totals within strata (between-stratum variability does not enter the variance); design degrees of freedom = number of PSUs - number of strata (Cochran, 1977; Lumley, 2004).
Effect on the result: SE, confidence interval and degrees of freedom; the point estimate does not change. If a cluster is also given, the PSU identity is nested as the (stratum, PSU) pair.
On the validation data: apistrat unstratified SE 9.58543, stratified by school type SE 9.53613; design df 197 instead of 199. apiclus1 (stratum dnum, no cluster): SE 5.80578.
Cluster / PSU
Concept: The cluster, or primary sampling unit (PSU), is the unit within which people are selected as a group rather than one by one (a school, village, neighbourhood). Because people in the same cluster resemble each other, each extra person brings less new information than an independent one, so clustering usually inflates the standard error.
Everyday example: If most of a school's 40 sampled pupils live in the same neighbourhood and grow up with similar rules, their screen times are close. These 40 pupils carry less information than 40 pupils chosen at random across the province; giving the cluster variable lets the standard error reflect this.
What it does: Sets the primary sampling unit. The variance is computed from PSU totals (ultimate cluster approximation, with replacement); the second-stage variance is not modelled separately.
Effect on the result: The SE usually grows because of within-cluster similarity; degrees of freedom = PSU - strata.
On the validation data: apistrat (no clusters) SE 9.58543; apiclus1 (cluster: dnum) SE 23.779, df 14 (15 - 1), DEFF 9.5348.
FPC (population)
Concept: The finite population correction accounts for the reduced uncertainty when a substantial part of the population is sampled; the variance is multiplied by (1 − n/N). With a small sampling fraction (e.g. under 5%) the correction barely changes the result.
Everyday example: If the rural stratum has 30 schools in total and 15 were sampled, half the population has already been observed and that stratum's contribution to the variance is halved. If 30 of the 600 urban schools were sampled, the correction is negligible.
What it does: Finite population correction: the population size N_h in the stratum (with clusters, the number of PSUs in the population); the variance is multiplied by (1 - n_h/N_h). Used when the sampling fraction is too large to ignore.
Effect on the result: The SE and confidence interval shrink; the without-replacement DEFF denominator is not affected.
On the validation data: apistrat without FPC SE 9.53613, with FPC 9.40894; apiclus1 cluster + no FPC 23.779, with FPC 23.5422.
Single-PSU stratum (No contribution (certainty, default), Centered at the average (adjust))
Concept: Design-based variance relies on how much the clusters within each stratum differ from one another; if a stratum contains only one cluster, this difference cannot be computed. This option decides whether such a stratum contributes nothing to the variance or whether its cluster is assessed against the average of all clusters, the more cautious approach.
Everyday example: If a mountain district forms a stratum of its own and only one school was sampled there, the data hold no information on how that district's schools differ. The cautious option adds an allowance to the standard error rather than ignoring this uncertainty.
What it does: Sets the variance contribution of a stratum with a single PSU. No contribution: the stratum does not enter the variance. Centered at the average: the stratum's PSU total is centred on the mean of all PSUs (more conservative; Lumley, 2010). A warning is written in every case.
Effect on the result: SE and confidence interval only when a single-PSU stratum exists.
On the validation data: apiclus1 (stratum dnum 413 has a single PSU): no contribution SE 5.80578; centred at the average SE 5.82806.
Subgroup (by)
Concept: Domain (subgroup) estimation computes the mean for a subpopulation while keeping the full design information. Filtering the data first and then analysing gives wrong standard errors, because it ignores that the subgroup's sample size is itself random and discards the structure of the design.
Everyday example: To get girls' and boys' screen time separately, sex is given as the subgroup. Copying only the girls into a separate file and analysing them loses design information, for example in schools where no girls happened to be sampled.
What it does: Computes subgroup (domain) means for each domain while fully preserving the design (filtering the data gives a wrong SE). For each domain the estimate, SE, confidence interval, CV and DEFF are given; the degrees of freedom are those of the full design.
Effect on the result: A separate row for each domain.
On the validation data: apiclus1, ell by school type: domain E 29.6944 (SE 1.41162), domain H 15 (SE 5.34707).
DEFF denominator (Automatic (default; based on the sum of weights), Without replacement (SPSS / R default), With replacement (WR; R "replace" / SAS))
Concept: The design effect (DEFF) is the ratio of the variance under the design to the variance of a simple random sample of the same size. The simple-random-sampling variance in the denominator can assume sampling with or without replacement; the choice affects only the DEFF value, not the mean or its standard error.
Everyday example: A DEFF of 2 means that 2000 pupils selected by the clustered design give about the same precision as 1000 chosen by simple random sampling. The without-replacement denominator makes a noticeable difference when a large share of all pupils in the province enters the sample.
What it does: Selects the simple random sampling variance in the denominator of the design effect: Automatic (with replacement if the sum of weights is about n, otherwise without), Without replacement (1 - n/N) s^2/n, With replacement s^2/n. Without replacement for population expansion weights, with replacement for relative weights (Kish, 1965).
Effect on the result: Only DEFF and DEFT.
On the validation data: apistrat api00, stratified design with FPC (stratum stype, SE 9.40894): without replacement 1.20446, with replacement 1.16557. Stratified design without FPC (SE 9.53613): without replacement 1.23724, with replacement 1.19729. The default 'Automatic' selects without replacement here (sum of weights 6194, n = 200). kigyo srs.w without replacement 1.0101, pps.w with replacement 1.97407, pps.w without replacement 1.98997.
Confidence Level (%90, %95, %99, Other (custom))
Concept: Sets how much assurance the confidence interval gives of covering the true population mean. In complex designs the interval uses design degrees of freedom, based on the number of clusters and strata rather than the number of respondents.
Everyday example: The provincial mean screen time might lie between 3.5 and 3.9 hours with 95% confidence; 99% confidence widens the range. If the 2000 pupils came from only 60 schools, it is those 60 schools, not the 2000 pupils, that determine how wide the interval is.
What it does: Sets the level of the confidence intervals (t distribution, design degrees of freedom).
Effect on the result: All intervals; SE and p do not change.
On the validation data: apistrat lower bound of the mean with FPC: 95% 643.732, 90% 646.738, 99% 637.815, 80% 650.189, 99.9% 630.856.
Additional output: Unweighted n and sum of weights (N̂)
Concept: The unweighted n is the number of respondents actually in the analysis; the sum of weights is the estimated size of the population they represent. Seeing both side by side shows how many people the estimate rests on and how large a population it generalises to.
Everyday example: If 2000 pupils are in the analysis but their weights sum to 85,000, these 2000 pupils represent roughly 85,000 middle-school pupils in the province.
What it does: Gives the valid unweighted number of observations and the sum of weights (estimated population size) per variable (and domain).
Effect on the result: Additional table only.
On the validation data: apiclus1 stratum H domain n = 14, sum of weights 473.858.
Additional output: Differences between domains (contrasts)
Concept: Gives the difference between subgroup means with its design-based standard error and tests whether it differs significantly from zero. Because two subgroups can share the same clusters, the difference should not be judged by eye from the two separate means' errors.
Everyday example: If urban pupils have 0.6 hours more screen time than rural pupils, this comparison gives the difference with a confidence interval such as 0.6 ± 0.3 hours and a p value, showing whether it is a real difference or sampling luck.
What it does: With a subgroup (by) selected, gives the difference, design-based SE, t, p and confidence interval for all pairs of domains.
Effect on the result: Additional table only.
On the validation data: apiclus1 ell, E - H difference 14.6944, SE 4.48074, p 0.00548172.
Ratio estimate (Y / X)
Concept: The ratio estimate gives the ratio of two population totals (e.g. total spending / total persons). Since both numerator and denominator are estimated from the sample, the ratio's standard error also accounts for how the two vary together.
Everyday example: You could ask for the ratio of pupils' total weekly screen time to their total weekly sleep: 'How many hours of screen time do pupils have for every hour of sleep?' A ratio of 0.4 means screen time is about two-fifths of sleep time.
What it does: Gives the ratio = sum(wy) / sum(wx) and its Taylor SE.
Effect on the result: Additional row only.
On the validation data: apiclus1 api_stu / enroll: ratio 0.849709, SE 0.0083863.
Quantiles (e.g. 0.25, 0.5)
Concept: Weighted quantiles give the point below which a given share of the population's values falls (e.g. the median, the quartiles). For skewed variables they summarise better than the mean, and their confidence intervals take the design into account.
Everyday example: If a few pupils spend 10 hours a day in front of a screen, the mean is pulled up. If the weighted median is 3 hours, you can say that half the pupils in the province have less than 3 hours of screen time a day.
What it does: Gives weighted quantiles and the Woodruff confidence interval (qrule = "math", interval.type = "mean").
Effect on the result: Additional row only.
On the validation data: apiclus1 api00 median 652, lower 561, upper 714.
Variance method (Taylor (default), Jackknife (JK1 / JKn), BRR (balanced half-samples), Fay BRR, Bootstrap (Rao-Wu), Supplied replicate weights) and related fields (Fay coefficient (rho), Bootstrap replicates, Random seed, Replicate weight columns, Replicate type (JK1, JKn, BRR, Fay, bootstrap, other), Scale, Replicate scales (rscales), Deviations from)
Concept: The standard error can be computed in two ways: Taylor linearisation is a formula-based approximation, while replication methods (jackknife, balanced repeated replication, bootstrap) form subsets or resamples of the data in line with the design, repeat the estimate in each, and derive the standard error from the spread of those estimates. Some national surveys are released with their own replicate weights.
Everyday example: In the jackknife, one school at a time is left out and the provincial mean recomputed; if the means are all very close, the result does not hinge on any single school and the standard error is small. If the ministry supplies replicate weights, those are used directly.
What it does: Computes the SE by resampling instead of Taylor: JK1 unstratified, JKn stratified; BRR and Fay BRR need 2 PSUs per stratum; bootstrap is the Rao-Wu method; supplied weight columns can also be used. Deviations are taken from the replicate mean (MerQur default) or can be taken from the full-sample estimate; df = rank(W) - 1.
Effect on the result: Only SE, t, confidence interval and degrees of freedom; the point estimates do not change. The additional outputs (domain contrasts, ratio, quantile) are computed with Taylor, as stated in the output.
On the validation data: apistrat JKn (with FPC) SE(api00) = 9.4089408027846, Rao-Wu bootstrap (B = 100, seed 2026) SE = 8.91258667858257.
Example on real data
Data: California API apistrat, R survey package
Variables: targets api00 (academic performance index) and enroll (enrolment), weight pw, stratum stype (school type)
- Question: In a stratified sample of California schools (strata by school type, weight pw), what are the population means of the academic performance index (api00) and the number of enrolled students (enroll)?
- Design: n = 200 schools, 3 strata (stype), no clusters (each school is a PSU); design df = 200 - 3 = 197; the SE is computed by Taylor linearization.
- Result: For api00 M̂ = 662.29 (SE = 9.54), 95% CI [643.48, 681.09]; for enroll M̂ = 595.28 (SE = 18.94), 95% CI [557.93, 632.63].
- Design effect: api00 DEFF = 1.24 (stratified stype design without FPC, SE 9.53613, without-replacement denominator; for this variable the design increases the variance by 24% relative to simple random sampling); enroll DEFF = 0.38 (stratification makes the estimate markedly more precise).
- Precision: The coefficients of variation are small (api00 1.44%, enroll 3.18%); the estimates are reliable.
- APA: api00: M = 662.29, SE = 9.54, 95% CI [643.48, 681.09], DEFF = 1.24; enroll: M = 595.28, SE = 18.94, 95% CI [557.93, 632.63], DEFF = 0.38 (N = 200, design df = 197).
Related concepts
Domain (subpopulation) estimationDesign effect (DEFF)Primary sampling unit (PSU) and cluster samplingTaylor linearization and replication variance methodsFinite population correction (FPC)Ratio estimator (Y/X)Stratified analysis (stratum variable)Sampling weight (design weight)
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.