MerQuR+Statistics Glossary

Statistics Glossary → Weighted Logistic Regression (Survey Logistic Regression)

Weighted Logistic Regression (Survey Logistic Regression)analysis

A logistic regression that relates the probability of a yes/no outcome to predictors in survey data collected under a complex sampling design, estimating the coefficients with sampling weights and the standard errors in a design-correct way.

Concept

Many outcomes of interest in surveys are binary: does the person smoke, are they vaccinated, are they overweight? Logistic regression models how the probability of such an event changes with the predictors. If the data were collected with a complex design, ordinary logistic regression may misrepresent the population and, because it ignores the similarity of people in the same cluster, understates the standard errors. Weighted logistic regression deals with both problems together.

The model writes the log-odds of the event as a linear combination of the predictors: log(p / (1 − p)) = b₀ + b₁x₁ + b₂x₂ + …. The coefficients are estimated by a weighted (pseudo) likelihood to which each observation contributes according to its weight, and the standard errors are computed from the stratum and cluster structure. The exponential of a coefficient, exp(b), is the odds ratio: OR = 1.5 says that a one-unit increase in the predictor multiplies the odds of the event by 1.5, holding the other predictors fixed; OR = 0.8 says it reduces them by 20%.

An odds ratio is not a ratio of probabilities; when the event is common, the two diverge markedly. Because the weighted likelihood is not a true likelihood, Wald tests are used to compare models, and criteria such as AIC should only be compared between models fitted with the same weights. The classification table and goodness-of-fit test add information on how well the model predicts probabilities. Causal interpretation needs support from design and theory.

When to use it

  • When the probability of a binary outcome is to be related to several predictors at population level, using a survey with weights, strata and clusters.
  • If the data come from a simple random sample, use ordinary binary logistic regression; if the outcome is continuous, use weighted linear regression.
  • If only the association between two categorical variables is asked, the weighted frequency's cross-table and Rao-Scott test are enough; if the outcome has more than two ordered categories, consider ordinal logistic models.

An everyday example

In the province-wide middle-school survey, 2000 pupils' height and weight are measured and each pupil is classified as overweight or not. Weighted logistic regression might find that, holding sleep time and breakfast habit fixed, each extra hour of daily screen time multiplies the odds of being overweight by about 1.2. Because pupils in the same school share similar canteen and physical-education conditions, the analysis takes this similarity into account and gives a confidence interval for the odds ratio that is wider than in an ordinary analysis, but more honest.

Parameters in MerQur

Predictors (X) and Target (Y, binary)

Concept: The dependent variable is a two-valued outcome (event present / absent); the predictors are the variables thought to affect its probability. Categorical predictors enter as dummy variables compared with a reference level.

Everyday example: Being overweight (yes/no) is the dependent variable; screen time, sleep time and breakfast habit are the predictors. The model shows how each predictor relates to the probability of being overweight, holding the others fixed.

What it does: Selects the binary (0/1) dependent variable and the predictors. If a strata, cluster or FPC column is also selected as a predictor, the model is built as the user requested.

Effect on the result: Coefficients, odds ratios, SE and Wald F.

On the validation data: apistrat, ell coefficient -0.00248964.

Weight (required)

Concept: The sampling weight shows how many people in the population each observation represents. In logistic regression each observation contributes to the likelihood in proportion to its weight, so the coefficients reflect the composition of the population rather than the sample.

Everyday example: If rural pupils were oversampled and overweight is less common in rural areas, an unweighted model can distort the province-wide relation. The weights give each pupil influence in line with their share of the population.

What it does: Sampling weight (inverse of the selection probability). Coefficients are computed by weighted (pseudo) likelihood and SE by Taylor linearization.

Effect on the result: Coefficients and SE.

On the validation data: apistrat ell coefficient -0.00248964 (the three programs the same).

→ Sampling weight (design weight)

Event (Y = 1) level

Concept: Logistic regression models the probability of whichever of the two categories is counted as the 'event' (1). Switching the event category flips the signs of the coefficients and turns odds ratios into 1/OR; what the model says is unchanged.

Everyday example: With 'overweight' as the event, the odds ratio for screen time is 1.2. With 'not overweight' as the event, the same relation appears as 1/1.2 ≈ 0.83: as screen time rises, the odds of being of normal weight fall.

What it does: Determines the level of Y treated as the event; with the automatic option, the larger value is taken for numeric Y and the last level in dictionary order for text Y.

Effect on the result: When the event changes, the sign of all coefficients flips (odds ratios are inverted); SE stays the same.

On the validation data: apistrat ell coefficient: -0.00248964 with event "Yes", 0.00248964 with event "No" (the three programs the same).

Stratum

Concept: Stratification divides the population into non-overlapping groups and samples each separately. In logistic regression, stratum information affects not the coefficients but the standard errors and the degrees of freedom of the tests.

Everyday example: Once the urban and rural strata are declared, the odds ratio for screen time is unchanged, but its confidence interval is computed in line with the design.

What it does: Computes the variance from the deviations of within-stratum PSU totals; design degrees of freedom = PSU - strata.

Effect on the result: SE, interval and degrees of freedom.

On the validation data: apistrat SE of the ell coefficient: unstratified 0.0134141, stratified by school type 0.0134669.

→ Stratified sampling (design stratum)

Cluster / PSU

Concept: The cluster (PSU) is the unit within which people are selected as a group. Because people in the same cluster share similar conditions, observations are not independent and standard errors usually grow.

Everyday example: A school whose canteen sells only packaged snacks may have a high share of overweight pupils. The shared situation of that school's pupils is reflected in the standard error once the school is declared as the cluster.

What it does: Defines the primary sampling unit; degrees of freedom = PSU - strata.

Effect on the result: SE and degrees of freedom.

On the validation data: apiclus1 (clustered by dnum) SE 0.0126995, df 14.

→ Primary sampling unit (PSU) and cluster sampling

FPC (population)

Concept: The finite population correction shrinks the variance by multiplying it by (1 − n/N) when a substantial part of the population is sampled. With a small sampling fraction its effect is negligible.

Everyday example: If half the rural schools entered the sample, the uncertainty from that stratum is reduced and the odds ratios' confidence intervals narrow slightly.

What it does: Finite population correction; the variance is multiplied by (1 - n_h/N_h).

Effect on the result: SE and interval become smaller.

On the validation data: apiclus1 cluster: without FPC 0.0126995, with FPC 0.0124427.

→ Finite population correction (FPC)

Single-PSU stratum (No contribution (certainty, default), Centered at the average (adjust))

Concept: If a stratum has only one cluster, the between-cluster variation within it cannot be computed. The option decides whether the stratum contributes nothing to the variance or whether a more cautious contribution is computed relative to the average of all clusters.

Everyday example: If one district stratum had only a single school sampled, the size of the differences between schools there cannot be known; the cautious option adds this unknown to the standard errors.

What it does: Determines the variance contribution of a stratum with a single PSU.

Effect on the result: SE only, when a single-PSU stratum is present.

On the validation data: apiclus1 no contribution SE 0.0158432; centered at the average 0.0158448.

→ Primary sampling unit (PSU) and cluster sampling

Also add the stratum to the model as a fixed effect (dummies)

Concept: Adding the stratum as a fixed effect gives each stratum its own intercept (baseline event probability), so the coefficients show the within-stratum relation. Otherwise the stratum is used only in computing the standard errors.

Everyday example: If both screen time and overweight are more common among urban pupils, the odds ratio found without the stratum partly reflects the urban-rural difference. With the stratum added, the odds ratio shows the relation among pupils in the same stratum.

What it does: Also adds the stratum to the model as dummy fixed effects; the default is off.

Effect on the result: Coefficients, SE and tests; enabled only when a stratum is selected.

On the validation data: apistrat ell coefficient: off -0.00248964, on -0.00783729 (the stratum dummies are added to the model). The coefficients are the same as in R and SPSS.

→ Stratified sampling (design stratum)

Confidence Level (%90, %95, %99, Other (custom))

Concept: Sets how much assurance the confidence intervals for coefficients and odds ratios give of covering the true population value. Odds-ratio intervals are built on the log scale and transformed back, so they are usually asymmetric.

Everyday example: If the odds ratio for screen time is 1.20, its 95% confidence interval might run from 1.05 to 1.37; since the interval excludes 1, the relation is significant. Asking for 99% confidence widens it.

What it does: Sets the level of the coefficient and odds ratio confidence intervals (t distribution).

Effect on the result: Intervals only.

On the validation data: apistrat mobility OR 95% CI [0.996, 1.134].

→ Confidence level and confidence interval

Reference levels

Concept: The odds ratios of a categorical predictor are computed relative to the chosen reference level. Changing the reference does not change model fit; it only determines which comparison can be read directly.

Everyday example: If breakfast frequency is 'never', 'sometimes' and 'every day', choosing 'every day' as the reference makes the odds ratios show directly the odds of being overweight for pupils who skip breakfast compared with those who eat it every day.

What it does: Sets the comparison level of a categorical predictor; if empty, the first level alphabetically.

Effect on the result: Dummy coefficients and odds ratios are re-expressed; model fit does not change.

On the validation data: apistrat stype: with base level E the H coefficient is -1.03694, with base level H the E coefficient is 1.03694.

→ Reference category (comparison level)

Interaction terms

Concept: An interaction term tests whether a predictor's effect on the event probability changes with the level of another predictor. If there is an interaction, each predictor's odds ratio cannot be interpreted on its own.

Everyday example: The effect of screen time on overweight may be stronger among pupils who skip breakfast. The 'screen time × breakfast' term tests whether this difference is significant.

What it does: Adds product terms in the form 'a*b'.

Effect on the result: Additional coefficients and the term test.

On the validation data: ell*meals coefficient 0.000326119 (the three programs the same).

→ Interaction effect

Subpopulation (domain) column and Domain value

Concept: When the model is fitted for a subpopulation, the other observations are not deleted but given zero weight, so the full design is kept for computing standard errors. Filtering the data beforehand produces wrong standard errors.

Everyday example: If the model is to be fitted for grade 8 only, 'grade = 8' is given as the subpopulation; pupils in other grades stay in the dataset with zero weight, and the schools' design information is not lost.

What it does: Fits the model only for the subpopulation with the given value; the design is fully preserved.

Effect on the result: Coefficients are computed from the subpopulation.

On the validation data: In the apistrat comp_imp = Yes subpopulation, the ell coefficient is -0.0400618.

→ Domain (subpopulation) estimation

Additional output: Classification table and Classification cut value

Concept: The classification table labels a person as 'event' when their predicted probability exceeds a cut-off and compares these predictions with the actual outcome. Sensitivity shows how many of the real events are caught, specificity how many non-events are correctly identified; the cut-off sets the balance between them.

Everyday example: With a cut-off of 0.5 the model labels only high-probability pupils as 'overweight' and may miss many real cases (low sensitivity). If a school health screening must not miss cases, the cut-off can be lowered to, say, 0.3; false alarms then increase.

What it does: According to the cut value (event if p-hat >= cut), it gives the observed and predicted classification table, the percentage correctly classified, the sensitivity and the specificity. The cut value is enabled only when the classification table is on.

Effect on the result: The classification table only.

On the validation data: apistrat, cut 0.5: weighted correct classification 0.827948, sensitivity 1. Specificity is 0 for cut 0.3 and 0.439115 for cut 0.8.

→ Classification table (confusion matrix)

Additional output: Archer-Lemeshow goodness-of-fit test

Concept: A goodness-of-fit test checks whether the probabilities the model predicts agree with the observed event frequencies. People are grouped by predicted probability and the observed and expected numbers of events are compared in each group; the Archer-Lemeshow test does this while accounting for the design. A significant result indicates poor fit.

Everyday example: If the model predicts an average 10% probability of overweight for a group of pupils, about 10% of them should actually be overweight. If in the highest-probability group the prediction is 40% but the actual rate is 60%, the model does not describe that group well.

What it does: Design-based goodness-of-fit test (Archer and Lemeshow, 2006): the Wald F for the hypothesis that the mean residuals in the weighted deciles of p-hat are zero. Tied probability predictions go to the same group.

Effect on the result: An additional test.

On the validation data: apistrat F = 3274.84; NHANES F = 4.99325 (7 groups).

→ Hosmer-Lemeshow goodness-of-fit test

Additional output: Save predictions and residuals to the dataset

Concept: Adds to the dataset the event probability the model predicts for each observation and the difference between the observed outcome and that prediction (the residual). These values can be used later, for example to group people by risk level or to find where the model performs poorly.

Everyday example: Once each pupil's predicted probability of being overweight is saved, you can map, for instance, which schools have the most pupils with a probability above 0.4.

What it does: Adds SVY_PRED_P (probability) and SVY_RESID columns.

Effect on the result: New columns only.

On the validation data: Row 1 probability 0.792397 (the three programs the same).

Variance method (Taylor (default), Jackknife (JK1 / JKn), BRR (balanced half-samples), Fay BRR, Bootstrap (Rao-Wu), Supplied replicate weights) and related fields (Fay coefficient (rho), Bootstrap replicates, Random seed, Replicate weight columns, Replicate type (JK1, JKn, BRR, Fay, bootstrap, other), Scale, Replicate scales (rscales), Deviations from)

Concept: The standard error can be computed by formula-based Taylor linearisation or by replication methods (jackknife, balanced repeated replication, bootstrap). In replication methods the model is refitted in each design-based subsample and the spread of the coefficients gives the standard error; subsamples that fail to converge can be dropped from the calculation.

Everyday example: In the bootstrap, schools are resampled and the model refitted hundreds of times. If in one replicate all the pupils who skip breakfast happen to be overweight, the model separates in that replicate and the coefficient does not settle on a finite value; such a replicate is left out and this is reported.

What it does: Computes the standard error by resampling instead of Taylor; the model is re-estimated in each replicate and the Wald tests are based on the replicate covariance. In the bootstrap a non-converging or separated replicate is excluded from the variance and a warning is written.

Effect on the result: SE, p, interval and Wald F; coefficient estimates do not change. Example: apistrat, 500 replicates and seed 12345: 1 replicate is excluded, mobility SE is 0.0332.

On the validation data: apistrat JKn Wald F 0.122424830393, denominator df 195, p 0.884840360414; nhanes BRR model Wald F 55.7640515423, df 10.

→ Taylor linearization and replication variance methods

Example on real data

Data: California API apistrat, R survey package (Lumley)

Variables: binary target sch_wide (school-wide target met; event Yes), predictors ell, meals, mobility; weight pw, stratum stype

  • Question: In a stratified school sample (weight pw, stratum stype), is meeting the school-wide target (sch_wide = Yes) related to ell, meals and mobility?
  • Model: Design-based weighted logistic regression (event = Yes), Taylor SE; t tests with t(194); n = 200.
  • Overall test: Wald F(3, 194) = 1.20, p = .310; adjusted Wald F(3, 195) = 1.19, p = .315. The predictors are not jointly significant.
  • Coefficients: mobility b = 0.061 (SE = 0.033), OR = 1.063, 95% CI [0.996, 1.134], p = .065 (borderline, not significant); meals (OR = 0.997, p = .737) and ell (OR = 0.998, p = .854) have no effect.
  • Model fit: McFadden R² = .029, Nagelkerke R² = .044; the explained variability is very small.
  • APA: Overall model (adjusted Wald): F(3, 195) = 1.19, p = .315; mobility: OR = 1.06, 95% CI [1.00, 1.13], p = .065.

Related concepts

Primary sampling unit (PSU) and cluster samplingHosmer-Lemeshow goodness-of-fit testStratified sampling (design stratum)Classification table (confusion matrix)Finite population correction (FPC)Taylor linearization and replication variance methodsOdds ratio (OR)Sampling weight (design weight)

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.