Statistics Glossary → Sample Selection and Truncated Regression (Heckman)
Sample Selection and Truncated Regression (Heckman)analysis
Methods that estimate regression coefficients while correcting the bias arising when the outcome is observed only for a selected subgroup (the Heckman model) or when observations beyond a certain limit never enter the data at all (truncated regression).
Concept
Some outcomes are seen only for certain people: wages only for those in work, exam scores only for those who sit the exam, loan interest only for those who get a loan. These people are not selected at random; people who expect a high wage, for example, are more likely to decide to work. If the regression uses only the observed cases, the coefficients are biased whenever unmeasured factors behind selection also affect the outcome. This is called sample selection bias.
The Heckman model treats two equations together. The selection equation (a probit) explains whether a person's outcome is observed, and the outcome equation explains the level of the observed outcome. The correlation between the errors of the two equations (ρ) shows how closely selection is linked to the outcome; if ρ = 0 there is no selection bias and ordinary regression is sufficient. The model can be estimated by maximum likelihood or in two steps (first a probit, then a regression including the inverse Mills ratio as an extra variable). For reliable estimation the selection equation should contain at least one variable that is absent from the outcome equation and affects selection but not the outcome directly (an exclusion restriction).
Truncated regression addresses a different situation: observations beyond the limit are absent from the data altogether; neither their outcomes nor their other characteristics are known, as in data recorded only for those scoring above a certain mark. Ordinary regression then usually shrinks the slopes towards zero; truncated regression corrects this by building the limit into the likelihood. Both methods are sensitive to the assumption that the errors are normally distributed.
When to use it
- When the outcome is observed only for people who passed through a selection process and this selection is suspected to be related to the outcome (wages of the employed, scores of exam takers, recovery of patients who stayed in treatment); use the Heckman model if a variable can be found that affects selection but not the outcome directly.
- If observations below or above a limit are entirely absent from the data, use truncated regression; if those observations are present but their outcome is recorded only at the limit (censoring), use a Tobit model.
- If selection is random, or the variables explaining selection are already in the outcome equation and ρ is close to zero, ordinary multiple regression is sufficient.
An everyday example
A district education office wants to study how weekly study time affects scores in the university entrance exam. But scores exist only for pupils who applied for the exam, and many who did not apply are pupils lacking confidence. A regression using only applicants may misstate the effect of study time. The Heckman model first models who applied (with variables such as family support and distance to the exam centre) and then estimates the score equation taking this selection into account. Distance to the exam centre acts as the exclusion restriction: it affects the decision to apply but not the score directly.
Parameters in MerQur
Outcome equation variables
Concept: The outcome equation contains the variables explaining the level of the observed outcome; their coefficients are effects purged of selection bias.
Everyday example: In the exam example the outcome equation explains the score with weekly study time and previous-year grade average. The coefficient for study time is estimated taking into account that applicants are a selected group.
What it does: Selects the explanatory variables of the outcome (wage) equation; categorical explanatory variables are converted to dummy variables.
Effect on the result: Determines the outcome equation coefficients and lambda.
On the validation data: educ, exper and expersq in Mroz; two-step: constant -0.5781033101, educ 0.1090655309 (SE 0.01552295487); lambda 0.03226186464, sigma 0.663628761, rho 0.04861432557.
Selection equation variables
Concept: The selection equation explains whether a person's outcome is observed. Having at least one variable in it that is absent from the outcome equation and affects selection but not the outcome directly (an exclusion restriction) is important for estimating the model reliably.
Everyday example: The selection equation explains the decision to apply with family support, grade average and distance to the exam centre. Distance does not affect the score but makes applying harder, so it appears only in the selection equation.
What it does: Selects the explanatory variables of the selection (probit) equation; at least one variable not in the outcome equation (exclusion restriction) is recommended, and a warning is given if there is none.
Effect on the result: Determines the selection probability and the inverse Mills ratio; identification rests on the exclusion restriction.
On the validation data: In Mroz educ, exper, expersq, nwifeinc, age, kidslt6, kidsge6 (two-step selection constant 0.2700767884, kidslt6 -0.8683285104).
Model (Heckman, maximum likelihood; Heckman, two-step (Heckit); Truncated regression)
Concept: Selects the estimation method. Maximum likelihood estimates the two equations together and is the most efficient; the two-step method first estimates the selection probability and then adds the inverse Mills ratio to the outcome equation as an extra variable, and it is somewhat less sensitive to the distributional assumption. Truncated regression is for data in which observations beyond the limit are entirely absent.
Everyday example: If both applicants and non-applicants are in the data but scores are known only for applicants, the Heckman model is used. If the data consist only of records of pupils scoring above 70, with nothing known about the others, truncated regression is needed.
What it does: ML estimates the selection and outcome equations jointly (sigma, rho, lambda by the delta method; rho interval on the atanh scale; LR and Wald independence tests). The two-step method uses OLS with the inverse Mills ratio after probit. Truncated regression is estimated by ML in a sample that never sees observations outside the limits.
Effect on the result: Changes the coefficients, SEs and tests. In two-step, rho is not clipped and a |rho| > 1 warning is given.
On the validation data: womenwk ML: rho 0.70350434731, sigma 6.00479456861, lambda 4.22439908372, log L -5178.30449805, LR 61.1986224344, Wald 508.43775894. Two-step: lambda 4.00161543465, rho 0.672839751196.
Target (Y)
Concept: The outcome variable to be modelled; in the Heckman model it is empty for unselected observations, while in truncated regression it contains only values within the limits.
Everyday example: The target is the university entrance exam score; in the rows of pupils who did not apply, this column is empty.
What it does: Selects the outcome variable (the truncated variable in truncated regression).
Effect on the result: Determines all outputs.
On the validation data: wage in womenwk.
Selection indicator ((y present = selected), 0/1 column)
Concept: A 0/1 variable stating which observations were selected, i.e. whose outcome was observed; it is the dependent variable of the selection equation.
Everyday example: Pupils who applied are coded 1 and those who did not are coded 0. Without a separate column, pupils with a recorded score are treated as having applied.
What it does: Specifies the 0/1 selection column meaning 1 = y observed; if left empty, rows with a filled y value are counted as selected.
Effect on the result: Determines the dependent variable of the selection equation; a non-binary indicator gives an explicit error.
On the validation data: In womenwk the 1343 working women are counted as selected; in Mroz the employment indicator is used.
Lower truncation limit and check box; Upper truncation limit and check box (active only in Truncated regression)
Concept: In truncated regression, states the limits within which the data lie: with a lower limit only observations above that value entered the sample, with an upper limit only those below it. Truncation differs from censoring; units beyond the limit do not appear in the data at all.
Everyday example: If a scholarship programme keeps files only for pupils scoring above 70, the lower truncation limit is 70. The model estimates the effect of study time taking into account that pupils below 70 exist but are not seen.
What it does: At the lower limit only observations with y greater than this value are in the sample (truncreg direction = "left"); at the upper limit only smaller ones (direction = "right"); both limits can be given together. Observations outside the limits are removed and reported.
Effect on the result: Changes the truncated likelihood and hence all coefficients and sigma.
On the validation data: laborsub from below at 0: log L -1200.9157373, kl6 -803.003720911 (SE 321.361050332), sigma 983.725679838. UCLA from below at 40: log L -591.30981349, langscore 0.712577503149.
Confidence Level (%90, %95, %99, Other (custom))
Concept: Sets the confidence level of the intervals for the coefficients and for the error correlation between the two equations (ρ); the higher the level, the wider the interval.
Everyday example: If the 95% confidence interval for ρ is [0.3, 0.7], the decision to apply is clearly related to the score, and a regression using only applicants would have been biased.
What it does: Determines the level of the coefficient and rho confidence intervals.
Effect on the result: Changes only the intervals.
On the validation data: At 95% the womenwk rho interval is 0.588534013409 to 0.790585042658.
Save the inverse Mills ratio to the dataset (check box)
Concept: The inverse Mills ratio is a term derived from each selected observation's probability of selection and used to correct selection bias; it is large for observations selected with low probability. The saved column can be used in later analyses.
Everyday example: A pupil who applied despite little family support and living far from the exam centre has a high inverse Mills ratio; they are an 'unexpected' applicant, and pupils like this contribute most to the selection correction.
What it does: Adds lambda = phi(z'gamma) / Phi(z'gamma) for the selected rows as a new column.
Effect on the result: Does not change the calculation.
Example on real data
Data: womenwk, Stata [R] heckman example data
Variables: target wage (hourly wage, missing for non-workers), outcome equation education and age, selection equation married, children, education, age
- Question: In data where the wage is observed only for working women (n = 2000, 1343 selected), what are the effects of age and education on the wage after correcting for selection bias?
- Selection equation (probit): Age (0.037), education (0.056), being married (0.445) and number of children (0.439) increase the probability of working (all p < .001).
- Outcome equation: Education B = 0.990 (SE 0.053, z = 18.59), age B = 0.213 (SE 0.021, z = 10.35); p < .001.
- Selection parameters: ρ = 0.704, 95% CI [0.589, 0.791]; σ = 6.005; λ = ρσ = 4.224 (SE 0.399).
- Independence test: LR χ²(1) = 61.20, p < .001; the selection and wage errors are correlated, so OLS on workers only would be biased.
- Fit: log L = -5178.30, AIC = 10376.61; outcome equation slopes Wald χ²(2) = 508.44.
- APA: Heckman selection model (ML), N = 2000 (1343 selected): ρ = 0.70, CI [0.59, 0.79], σ = 6.00; independence LR χ²(1) = 61.20, p < .001.
Related concepts
Delta methodLikelihood ratio test (LRT)Sample selection bias (Heckman)Inverse Mills ratioTruncationLink functionMaximum likelihood (ML) estimationModel identification (std.lv, unit loading)
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.