MerQuR+Statistics Glossary

Statistics Glossary → Quantile Regression

Quantile Regressionanalysis

A regression that estimates how chosen quantiles of the outcome, such as the median, the lower quartile or upper percentiles, rather than its mean, change with the predictors, showing whether an effect differs across parts of the distribution.

Concept

Ordinary regression answers a single question: how does the mean of the outcome change as a predictor changes? Yet an effect can differ across parts of the distribution. As income rises, average food spending rises, but among high-spending households the increase may be steeper; a training programme may lift low-achieving pupils markedly while hardly affecting high achievers. Quantile regression makes such differences visible by estimating a separate regression line for each chosen part of the distribution.

The τ quantile (0 < τ < 1) is the point below which a share τ of the values lies; τ = 0.5 is the median. Quantile regression sums not the squared residuals but their absolute values with asymmetric weights: for τ = 0.9, residuals of observations above the line are penalised more heavily, so the line settles where about 90% of observations lie below it. The slope for each quantile shows how much that quantile changes when the predictor rises by one unit. If the slopes are the same across quantiles, the predictor merely shifts the distribution; if they differ, it also changes the distribution's spread or shape.

The method needs no assumption that errors are normal or have equal variance, and it is more robust to outliers in the outcome than mean regression. On the other hand, estimates at extreme quantiles (e.g. τ = 0.05 or 0.95) rest on few observations, are uncertain and need large samples. The coefficients concern conditional quantiles; individual readings such as 'people in the upper quantile' should be avoided, and the result read as 'the upper part of the conditional distribution'.

When to use it

  • When an effect is suspected to differ in the lower, middle and upper parts of the distribution, or when the interest lies in the tails rather than the mean: income and spending, wage inequality, growth-chart percentiles, worst-case waiting and delivery times.
  • If only the average effect is asked and the errors are roughly normal with equal variance, ordinary linear regression is sufficient and more precise.
  • If the aim is only to stop a few outliers distorting the mean line, robust regression is another option; the median (τ = 0.5) line of quantile regression can also serve this purpose.

An everyday example

A food-delivery company examines how delivery time changes with the distance between restaurant and customer. Mean regression shows that each extra kilometre adds about 3 minutes. Quantile regression might give a more detailed picture: for fast deliveries (τ = 0.1) each kilometre adds about 2 minutes, while for the slowest deliveries (τ = 0.9) it might add 6, because traffic and hard-to-find addresses pile up on long trips. When the company promises customers that '90% of deliveries arrive within this time', it should look at the τ = 0.9 line, not the mean line.

Parameters in MerQur

Explanatory variable(s) and Target column

Concept: The dependent variable is the outcome whose quantiles are studied; the predictors are the variables expected to explain how those quantiles change. Categorical predictors enter as dummy variables compared with a reference level.

Everyday example: Delivery time is the dependent variable; distance and whether it is raining (yes/no) are the predictors. If the rain coefficient is large at τ = 0.9, rain delays most of all the deliveries that are already slow.

What it does: Selects the dependent and explanatory variables; categorical variables are expanded into dummy variables.

Effect on the result: All estimates.

On the validation data: engel, tau = 0.5 slope 0.5601806; intercept 81.4822474.

Quantiles (CSV)

Concept: Each τ value defines a part of the distribution to be estimated: τ = 0.25 is the lower quartile, τ = 0.5 the median, τ = 0.75 the upper quartile. Estimating several quantiles together shows how the effect changes across the distribution.

Everyday example: If the company chooses τ = 0.1, 0.5 and 0.9, it sees the effect of distance side by side for fast, typical and slow deliveries. If it cares only about customer complaints, it can choose a single upper quantile such as τ = 0.95.

What it does: Gives the tau values to be estimated, separated by commas (between 0 and 1); the default is 0.25, 0.50, 0.75. Inputs outside (0, 1) or non-numeric are discarded with a warning; if none is valid, the default is applied.

Effect on the result: The estimated quantiles.

On the validation data: engel income slope for tau = 0.25 / 0.5 / 0.75 is 0.4741 / 0.5602 / 0.6440. The input "0.5, 1.2, abc, 0" reduces to 0.5 only.

→ Percentiles (quantiles) and percentile definitions

SE / CI method (Robust kernel, nid, ker, boot)

Concept: The standard error of quantile regression coefficients depends on the density of the data around that quantile and can be estimated in different ways. The robust kernel method uses a single density estimate; the nid and ker methods allow the density to vary with the predictors, i.e. unequal variance; the bootstrap resamples the data and works without distributional assumptions.

Everyday example: If delivery times are close together for short distances and widely spread for long ones, the variance grows with distance. The nid or ker method then shows the uncertainty of the long-distance estimates more realistically.

What it does: Selects the standard error method: robust (Hall-Sheather bandwidth, Epanechnikov kernel, sandwich with a single density f0), nid (Hendricks-Koenker), ker (Powell kernel) and boot (xy bootstrap, B = 200, seed 42). Coefficients are the exact LP solution in every method; p and intervals use t(n - p). For heteroskedastic data nid or ker, and for small n boot, is recommended (Koenker, 2005).

Effect on the result: SE, p and confidence intervals; coefficients do not change.

On the validation data: engel tau = 0.25 intercept SE: robust 18.550; nid 21.392; ker 24.164.

→ Robust (sandwich) standard errors (HC0-HC3)

Test equality of slopes across quantiles (Wald)

Concept: This test checks whether a predictor's slope is the same at the chosen quantiles. If equality is rejected, the predictor does not merely shift the distribution but also changes its spread or shape.

Everyday example: If distance has a slope of 2 minutes at τ = 0.1 and 6 minutes at τ = 0.9, the test says whether this difference can be explained by chance. If it is significant, distant deliveries not only take longer but also become far less predictable.

What it does: Tests whether the effect differs across quantiles (Koenker-Bassett Wald test, nid covariance): a joint test and a separate test for each slope. At least two quantiles are required.

Effect on the result: The test table only.

On the validation data: For salary years, W = 3.4026, chi-square p = 0.1825; joint F(6, 1371) = 5.5276.

→ Wald test and Wald confidence interval

Chart (Coefficient curves, Quantile process + CI bands, Fit plot)

Concept: Coefficient curves and the quantile process show how a predictor's coefficient changes as τ goes from 0 to 1, with a confidence band and the ordinary regression coefficient for reference. The fit plot draws the different quantile lines over the data points.

Everyday example: The chart shows the distance coefficient rising steadily from fast deliveries to slow ones, with ordinary regression's single value of 3 minutes sitting somewhere in the middle of the curve. The fit plot shows the τ = 0.9 line climbing ever more steeply at long distances.

What it does: Draws coefficient curves, the quantile process on a tau = 0.05-0.95 grid (confidence band with the selected SE and the OLS interval) or the scatter plot. When the process is selected, the coefficients and intervals on the grid are added to the text.

Effect on the result: The process option adds numerical output to the text; the others are graphics only.

On the validation data: The process plot shows the coefficient and confidence interval for each tau, alongside the least squares coefficient and interval.

Save quantile predictions and residuals to the dataset

Concept: Adds to the dataset, for each quantile, the value the model predicts and the difference between the observed value and the prediction. In a sound model for quantile τ, roughly a share τ of the observations should have negative residuals.

Everyday example: If the τ = 0.9 prediction is saved for each order, orders exceeding that time can be flagged automatically. If the model works well, about 10% of orders exceed the limit.

What it does: Adds QR_PRED_qXX and QR_RESID_qXX columns for each tau.

Effect on the result: New columns only.

On the validation data: The added columns hold the fitted value and residual for each engel observation.

Confidence Level (%90, %95, %99, custom)

Concept: Sets how much assurance the coefficients' confidence intervals give of covering the true value. A higher level gives a wider interval.

Everyday example: If the distance slope at τ = 0.9 is 6 minutes, you can say with 95% confidence that the true slope lies between, say, 4.8 and 7.2 minutes; asking for 99% widens the interval.

What it does: Sets the confidence interval level (t(n - p)).

Effect on the result: Intervals; in MerQur and quantreg the SE does not change because alpha is fixed at 0.05 in the Hall-Sheather bandwidth.

On the validation data: engel nid, 90%: the interval changes, the SE does not.

→ Confidence level and confidence interval

Example on real data

Data: Engel 1857 food expenditure, quantreg package

Variables: target foodexp (food expenditure), explanatory income (household income)

  • Question: In Belgian households, does income (income) affect food expenditure (foodexp) in the same way in the lower, middle and upper parts of the distribution?
  • Model: Quantile regression, τ = .25, .50, .75; coefficients are the exact LP solution; SE is the MerQur default "robust" (Hall-Sheather bandwidth, Epanechnikov kernel); N = 235.
  • Result: income slope 0.474 at τ = .25 (SE = 0.017), 0.560 at τ = .50 (SE = 0.013), 0.644 at τ = .75 (SE = 0.011); all p < .001.
  • Interpretation: The slope grows with the quantile (heterogeneous effect): in high-spending households the effect of income on food expenditure is stronger. The OLS slope (0.485) hides this difference.
  • Model fit: Pseudo R² τ = .25: .554, τ = .50: .621, τ = .75: .697.
  • Limitation: The "robust" SE is based on a sandwich with a single density f0; the nid SEs are wider (τ = .25 intercept: 21.39 and 18.55).
  • APA: Quantile regression (N = 235): τ = .25: b = 0.474, SE = 0.017; τ = .50: b = 0.560, SE = 0.013; τ = .75: b = 0.644, SE = 0.011; all p < .001.

Related concepts

Percentiles (quantiles) and percentile definitionsWald test and Wald confidence intervalScatter plotStandard errorRobust (sandwich) standard errors (HC0-HC3)Bandwidth (kernel smoothing)HeteroscedasticityOrdinary least squares (OLS)

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.