MerQuR+Statistics Glossary

Statistics Glossary → Design effect (DEFF)

Design effect (DEFF)concept

In a complex sample design (clusters, strata, weights), the ratio of an estimate's variance to the variance expected from a simple random sample of the same size.

Real surveys usually do not use simple random sampling; they select clusters (neighbourhoods, schools) and sample people within them, divide the population into strata and use unequal weights. Because people in the same cluster resemble each other, each extra person brings less new information and the variance of the estimates grows. DEFF = Var_design / Var_simple random. A DEFF of 2 means the estimate's variance is twice that under simple random sampling; the effective sample size is n / DEFF. For cluster sampling, Kish's approximation is DEFF ≈ 1 + (m - 1)·ρ (m being the cluster size and ρ the intraclass correlation).

Clustered designs and variable weights usually push DEFF above 1, while well-chosen stratification can bring it below 1. DEFF varies from estimate to estimate: in the same survey it may be 1.2 for one variable and 3 for another that is very similar within clusters. Standard errors that ignore the design are too small by a factor of the square root of DEFF. Software packages may differ in whether the simple-random-sampling variance in the denominator is computed with or without replacement (the finite population correction).

An everyday example

To estimate how many pupils wear glasses, a researcher selects 20 schools and measures 50 pupils in each (1,000 pupils). Because pupils in the same school share similar socioeconomic conditions, the intraclass correlation is 0.04. Then DEFF ≈ 1 + 49 x 0.04 = 2.96: this sample of 1,000 pupils carries about as much information as a simple random sample of roughly 340.

In MerQur

In Weighted Mean and Weighted Frequency, the 'DEFF denominator' option (Automatic, Without replacement, With replacement) sets how the simple-random-sampling variance is computed. In Survey-PHREG, 'Additional output: Design effect (DEFF)' adds a DEFF column to the coefficient table.

Analyses where this concept appears

Weighted Mean (Survey Means)Weighted Frequency (Survey Freq)Survey-PHREG (Weighted Cox)

Related concepts

Primary sampling unit (PSU) and cluster samplingSampling weight (design weight)