Statistics Glossary → Instrumental Variables Regression (2SLS)
Instrumental Variables Regression (2SLS)analysis
A regression method that, when an explanatory variable is correlated with the error term (endogenous), consistently estimates its effect on the outcome by using an instrumental variable that affects the explanatory variable but influences the outcome only through it.
Concept
Ordinary regression assumes that the explanatory variable is uncorrelated with the error term. This fails in three situations: when an unmeasured variable affects both the explanatory variable and the outcome (omitted variable), when the outcome also affects the explanatory variable (simultaneity, e.g. price and demand), or when the explanatory variable is measured with error. The explanatory variable is then endogenous, and the least-squares coefficient systematically misestimates the true effect; this bias does not disappear as the sample grows.
An instrument (Z) must satisfy two conditions: it must be strongly related to the endogenous explanatory variable (X) (relevance), and it must affect the outcome (Y) through no route other than X (the exclusion restriction). Two-stage least squares (2SLS) first predicts X from the instruments and exogenous variables, then relates Y to this predicted part of X, which comes only from the instruments and is uncorrelated with the error. LIML reaches the same goal with a different estimator and is less biased when the instruments are weak or numerous. When several interrelated equations (such as supply and demand) are to be estimated together, system methods such as SUR and 3SLS are used.
The result is only as credible as the instruments. Weak instruments (a first-stage F below roughly 10) make the estimates biased and the standard errors unreliable. The exclusion restriction cannot be tested directly; it needs a theoretical justification. When there are more instruments than endogenous variables, overidentification tests (Sargan, Hansen J) partly check whether the instruments are consistent with one another. The Wu-Hausman test checks whether the variable is really endogenous, i.e. whether an instrumental-variables method is needed.
When to use it
- When the causal effect of an explanatory variable on the outcome is wanted but that variable is tied to the error term through unmeasured factors, simultaneity or measurement error, and a valid instrument can be found (returns to education, price elasticity, supply and demand estimation).
- If the explanatory variable is exogenous (Wu-Hausman not significant), ordinary least squares gives more precise results; if the unobserved factors are constant over time and panel data are available, fixed-effects panel regression can also remove the bias.
- If a before-after comparison of treated and control groups is possible, use difference-in-differences; if the endogeneity arises only from observed variables, propensity score methods can be considered.
An everyday example
An ice-cream chain wants to know how much a price increase reduces sales. But because demand rises on hot days, prices are raised on those days too, so the raw relationship between price and sales can look misleadingly like 'higher prices, higher sales'. The cost of raw milk is used as the instrument: when milk becomes more expensive the price of ice cream rises, but the cost of milk does not directly change how much customers want ice cream. Using only the part of the price change that comes from milk costs, 2SLS might estimate that a 10% price increase cuts sales by about 8%.
Parameters in MerQur
Estimator (2SLS (two-stage least squares) / LIML (limited-information maximum likelihood))
Concept: Selects the method for estimating the effect of the endogenous variable with instruments. 2SLS is the most common estimator; LIML is less biased than 2SLS when the instruments are weak or numerous. With exactly one instrument for one endogenous variable, the two give the same result.
Everyday example: In the ice-cream example, if milk cost is the only instrument, 2SLS and LIML give the same price effect. If several weak instruments such as milk, sugar and energy costs are used together, LIML can offer a more reliable estimate.
What it does: Estimates the effect of the endogenous variable with instruments. 2SLS uses the first-stage predictions in the second stage; LIML is a k-class estimator (κ >= 1) and is less biased than 2SLS with weak, many instruments.
Effect on the result: Coefficients, SE, R², σ and Sargan change; the first stage, weak-instrument F and Wu-Hausman stay the same; κ is written for LIML.
On the validation data: hsngval 0.002240 (SE 0.000339) -> LIML 0.002669 (0.000430), κ = 1.256906; R² 0.599 -> 0.490. In an exactly identified model (mroz, one instrument) κ = 1 and LIML equals 2SLS.
Standard errors (Classical / HC0 / HC1 / HC3 / Clustered)
Concept: Determines how the uncertainty of the coefficients is computed. The classical standard error assumes equal error variance; HC types are robust to unequal variance, and the clustered standard error to correlation among observations in the same cluster. The coefficients do not change.
Everyday example: If sales fluctuate much more in large branches than in small ones, the variance is unequal and HC standard errors are more reliable. If different weeks of the same branch resemble one another, standard errors clustered by branch are used.
What it does: Coefficient covariance: classical (homoskedastic), heteroskedasticity-robust HC0, HC1, HC3 and clustered.
Effect on the result: SE, t, p, CI, Wald F, weak-instrument F and Wu-Hausman change; coefficients, Sargan and Hansen J do not.
On the validation data: hsngval SE: classical 0.000339; HC0 0.000672; HC1 0.000693; HC3 0.000947; clustered (division, 9 clusters) 0.000407. Weak-instrument F: 13.30, 12.97, 11.42, 7.55 (F < 10 warning with HC3), 93.05.
Cluster column
Concept: Specifies the grouping used for the clustered standard error; errors of observations within the same cluster are allowed to be related. With few clusters these standard errors become unreliable.
Everyday example: If the data are weekly sales from 30 branches, the cluster column is the branch. With only 4 branches, the clustered standard error could not give a meaningful measure of uncertainty.
What it does: The group variable of the clustered SE (e.g. state, school). Used only when the SE type is Clustered; an error is given if it is empty or is the y, endogenous or instrument column.
Effect on the result: The clustered SE and diagnostics change. If the number of clusters is at most the number of parameters, a warning is given and undefined joint tests are shown as "-".
On the validation data: division (9 clusters) hsngval SE 0.000407; CigarettesSW state (48 clusters) lrprice SE 0.1809 (classical 0.1552).
Categorical (factor) variables
Concept: Expands numerically coded categorical variables into dummies; these can be used as endogenous, instrumental or exogenous variables.
Everyday example: If the region of each branch is coded 1 to 4 and regions buy from different milk suppliers, region dummies can also serve as instruments that shift the price.
What it does: Treats numerically coded columns (e.g. region 1-4) as categorical and expands them into dummies with the first level as baseline; categorical columns can be used in the endogenous, instrument and exogenous selectors.
Effect on the result: The selected column enters the model as dummies.
On the validation data: Instrument region + Factors = region: region is expanded into dummies with the first level omitted (hsngval 0.0022398; Wu-Hausman 15.9067; Sargan 11.2877).
Confidence Level (90% / 95% / 99% / Other (custom))
Concept: Sets the confidence level of the intervals for the coefficients; the higher the level, the wider the interval.
Everyday example: The price elasticity might lie between -1.1 and -0.5 with 95% confidence; at 99% the interval widens. With weak instruments, these intervals may look narrower than they should.
What it does: The level of the coefficient interval (t(n - k)).
Effect on the result: Only the interval limits change.
On the validation data: hsngval: 90% [0.001671, 0.002808]; 95% [0.001558, 0.002921]; 99% [0.001330, 0.003149].
Estimation type (Single equation (2SLS / LIML) / System: SUR (seemingly unrelated regression) / System: 3SLS (three-stage least squares)), System equations, Residual covariance denominator (sqrt((T - k_i)(T - k_j)) (systemfit) / T (uncorrected; Stata sureg/reg3)), Iterated (ITSUR / IT3SLS)
Concept: Estimates several interrelated equations together. SUR makes the estimates more precise by exploiting correlation between the errors of separate equations; 3SLS adds the use of instruments for endogenous variables. The small-sample correction in computing the error covariance can change the results slightly.
Everyday example: The ice-cream market has both a demand equation (sales depending on price and temperature) and a supply equation (production depending on price and milk cost), and price is endogenous in both. 3SLS estimates the two equations together, giving both relationships at once.
What it does: FGLS estimation of a multi-equation system using cross-equation error correlation: SUR (OLS per equation, then block GLS) and 3SLS (2SLS per equation, then block GLS). The denominator of Σ̂ can be the geomean √((T - k_i)(T - k_j)) or T.
Effect on the result: The choice of Σ̂ denominator changes the coefficients and SEs; programs using the same denominator should be compared.
On the validation data: Kmenta (n = 20): coefficients, SEs and diagnostics are reported in four specifications; the 3SLS supply coefficients are 52.1972, 0.2286, 0.2282, 0.3611.
Custom contrast (L)
Concept: Tests a researcher-defined linear combination of the coefficients, for example whether two coefficients are equal or whether their sum equals a given value.
Everyday example: If the chain wants to ask 'is the effect of price on sales equal in size to the effect of advertising?', the difference between these two coefficients is tested with a custom contrast.
What it does: Tests a linear contrast (L β) of the model coefficients; rows are entered with coefficient names ('groupB - groupA') or as a list of numbers. Estimate, SE, test and CI come from the full covariance of the selected SE type; with several rows a joint Wald test is given.
Effect on the result: Only an extra contrast section is added; the model table does not change. Changing the SE type changes the contrast SE and test. The field is disabled in system mode.
On the validation data: Kmenta ivreg(Q ~ P + D | D + F + A): estimates 0.0704352565722 (SE 0.0789248288742) and -0.557548332124, joint F = 23.8065150421; with HC1 F = 29.2491776556.
Example on real data
Data: Stata hsng, [R] ivregress
Variables: dependent rent, endogenous hsngval (house value), instruments faminc (family income) and region dummies reg2-reg4, exogenous pcturban (percent urban)
- Question: How is the effect of house value (hsngval) on rent estimated when house value is endogenous, using family income and region dummies as instruments?
- Assumption: In the first stage the joint test of the excluded instruments is F(4, 44) = 13.30, p < .001 (partial R² = .55); the instruments are strong (F >= 10). Wu-Hausman F(1, 46) = 15.91, p < .001; hsngval is endogenous and the OLS estimate (b = 0.0015) is inconsistent. Sargan χ²(3) = 11.29, p = .010; at least one instrument may be invalid (robust Hansen J χ²(3) = 6.84, p = .077, borderline).
- Result: With 2SLS hsngval b = 0.0022 (SE = 0.0003), t(47) = 6.61, p < .001, 95% CI [0.0016, 0.0029]; a 1000-dollar increase in house value raises monthly rent by 2.24 dollars on average. pcturban is not significant (b = 0.08, p = .793).
- Effect size: R² = .60, σ = 22.86; Wald F(2, 47) = 42.66, p < .001.
- APA: 2SLS (N = 50); hsngval: b = 0.002, SE = 0.000, t(47) = 6.61, p < .001; first stage F(4, 44) = 13.30; Wu-Hausman F(1, 46) = 15.91, p < .001; Sargan χ²(3) = 11.29, p = .010.
Related concepts
Two-stage least squares (2SLS) and LIMLSeemingly unrelated regression (SUR) and 3SLSInstrumental variable and endogeneityHausman testCluster-robust (clustered) standard errorsContrast and custom contrast (L)Robust (sandwich) standard errors (HC0-HC3)Model identification (std.lv, unit loading)
The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.