MerQuR+Statistics Glossary

Statistics Glossary → Ordinal Logistic Regression

Ordinal Logistic Regressionanalysis

A regression model that relates the tendency of an ordered categorical outcome, such as low-medium-high or Likert responses, to fall in higher categories to predictors, giving the effects as odds ratios.

Concept

Many measurements are ranks rather than numbers: satisfaction level, disease stage, product quality grade. The categories are ordered, but the distances between them cannot be assumed equal; the gap between 'very satisfied' and 'satisfied' need not equal the gap between 'satisfied' and 'neutral'. Scoring the categories 1, 2, 3 and fitting a linear regression assumes equal spacing, while ignoring the order and fitting a multinomial model wastes information. Ordinal logistic regression uses the order and only the order.

The most common form is the cumulative logit (proportional odds) model. It writes the probability of the outcome being in a given category or below with a separate intercept for each threshold but a common slope across all thresholds: logit P(Y ≤ j) = θⱼ − (β₁x₁ + β₂x₂ + …). Behind it is the idea that the categories arise from cutting a latent continuous variable at thresholds. exp(β) shows how many times a one-unit increase multiplies the odds of being in a higher category, and this ratio is assumed to be the same whichever threshold is considered.

This shared ratio is called the proportional odds (parallel lines) assumption and is checked with the test of parallel lines. If it fails, a partial proportional odds model, in which the offending predictors get slopes that vary by threshold, can be used. Pseudo-R² values are usually low and should not be read like R² in linear regression.

When to use it

  • When the outcome has three or more ordered categories (a Likert item, a stage, low-medium-high risk) and you want to study how this order relates to predictors.
  • If the categories have no natural order use multinomial logistic regression; with only two categories use binary logistic regression.
  • If the proportional odds assumption is clearly violated, consider a partial proportional odds model or multinomial logistic regression; for a near-continuous scale score built from many items, linear regression may also be reasonable.

An everyday example

A hospital asks 400 discharged patients how satisfied they were with the service: 'dissatisfied / neutral / satisfied'. Waiting time in the emergency department and whether the patient had a single room are also recorded. If the ordinal model gives OR = 0.7 for each extra hour of waiting, then, with the other factor fixed, a patient who waited one hour longer has about 30% lower odds of being in a higher satisfaction category; this ratio is assumed to be the same for 'neutral or above' as for 'satisfied'.

Parameters in MerQur

Predictors (X) and Target (Y, ordinal)

Concept: The target is an outcome with at least three ordered categories. For J categories, J − 1 thresholds and a single slope per predictor are estimated.

Everyday example: In the hospital example the target is the 'dissatisfied / neutral / satisfied' response and the predictors are waiting time and room type. With three categories the model estimates two thresholds.

What it does: Selects an ordered dependent variable with at least 3 categories and the predictors.

Effect on the result: J - 1 thresholds and p slopes are estimated.

On the validation data: apply (0/1/2, n = 400), ses (1/2/3, n = 200), rating (1-5, n = 72), Sat (Low/Medium/High, n = 1681).

Category order

Concept: Specifies the order of the categories from lowest to highest. Reversing the order flips the signs of the coefficients but the model carries the same information; an incorrect order (e.g. alphabetical) makes the model meaningless.

Everyday example: If the answers were stored as 'low, medium, high' satisfaction, alphabetical sorting would give 'high, low, medium', which is meaningless. Stating the order explicitly as 'low, medium, high' ensures that the waiting-time coefficient is read in the right direction.

What it does: Sets the low-to-high order of the categories, comma-separated; cumulative probabilities and thresholds follow this order. Writing it explicitly is safest for text categories.

Effect on the result: Reversing the order flips the slope signs and gives the thresholds as -θ in reverse order; LL stays the same. "unlikely / somewhat likely / very likely" and Low/Medium/High are recognised automatically; a label not in the data is skipped with a warning, a missing level gives an error.

On the validation data: Reversed order: pared 1.0477 -> -1.0477, thresholds 2.2033 / 4.2988 -> -4.2988 / -2.2033. Automatic recognition matches the UCLA output (pared 1.0477, gpa 0.6157).

Categorical (factor) variables

Concept: Makes a numerically coded categorical predictor enter the model with separate coefficients per level (dummy variables) rather than a single slope.

Everyday example: If the ward a patient stayed on is coded 1, 2, 3, these numbers are not measurements. As a factor, each ward's odds of higher satisfaction are given separately relative to the reference ward.

What it does: Turns numerically coded categorical predictors into factors (numeric order, first level is the baseline). Empty means no change.

Effect on the result: Instead of a single continuous slope, (levels - 1) dummy coefficients are estimated.

On the validation data: Cheese data, "Additive": -2LL 711.348, LR 148.4539 (df 3), AIC 733.348, OR 0.035, 0.181, 5.017. With the field empty Additive is continuous (-2LL 844.271).

→ Dummy (indicator) variable coding

Link function (Cumulative Logit (default), Cumulative Probit, Complementary Log-Log, Negative Log-Log, Cauchit)

Concept: Determines which distribution the latent continuous variable behind the categories follows. Logit is the most common choice and allows the odds-ratio interpretation; probit assumes a normal distribution; other links suit responses piled up in the lower or upper categories.

Everyday example: If most patients said 'satisfied' and very few fell in the lower categories, responses are piled at the top and a link other than logit may fit slightly better. For interpretation, logit with its odds ratios is still the easiest.

What it does: Selects the latent variable distribution: logit (logistic), probit (normal), cloglog (for piling in low categories), negative log-log (for piling in high categories), cauchit (heavy-tailed).

Effect on the result: Coefficients are on a different scale and LL differs.

On the validation data: apply (LL / pared): logit -358.5124 / 1.0477; probit -358.7476 / 0.5981; cloglog -359.7491 / 0.5166; loglog -359.3460 / 0.7486; cauchit -360.7399 / 1.0081.

→ Link function

Confidence Level (%90, %95, %99, Other (custom))

Concept: Sets the confidence level of the intervals for the slopes, thresholds and odds ratios; the higher the level, the wider the interval.

Everyday example: If waiting time has OR = 0.7, its 95% interval might be [0.58, 0.85]; at 99% the interval widens, but as long as it excludes 1, the conclusion that waiting lowers satisfaction holds.

What it does: Sets the Wald interval (b ± z·SE) for slopes and thresholds and the OR interval (exp).

Effect on the result: Only the intervals change. The output notes that the SEs come from the observed information matrix.

On the validation data: pared OR interval: 90% [1.841, 4.414]; 95% [1.693, 4.800]; 99% [1.438, 5.654].

→ Confidence level and confidence interval

Extra output: Test of parallel lines

Concept: Tests the proportional odds assumption, i.e. whether a predictor's effect is the same at every threshold. A significant result indicates that at least one predictor's effect changes from threshold to threshold.

Everyday example: If waiting time strongly affects the split between 'dissatisfied' and 'neutral' but has no effect on the split between 'neutral' and 'satisfied', the assumption fails. The test says whether such a difference can be explained by chance.

What it does: Tests the proportional odds assumption: are the slopes the same at all thresholds. Gives an LR against the general threshold-specific model (df = (J - 2) × p) and an LR per variable.

Effect on the result: An extra table.

On the validation data: apply 4.0138 (df 3, p .260); wine overall 3.762 (df 6), temp 3.1750 / contact 0.5667.

→ Proportional odds (parallel lines) assumption

Extra output: Pearson / Deviance goodness of fit

Concept: Compares observed and model-expected category frequencies for groups of observations sharing the same combination of predictor values. It is meaningful with categorical predictors; with continuous predictors the groups become too small and the test is unreliable.

Everyday example: If the predictors are only 'room type' and 'ward', each group contains enough patients and the observed-expected comparison is meaningful. When waiting time is entered in minutes, the groups shrink to single patients.

What it does: Gives Pearson and Deviance over covariate patterns. Meaningful when the number of patterns is small (categorical predictors); not valid with a continuous predictor.

Effect on the result: Extra rows.

On the validation data: wine 4 patterns: Pearson 3.5719, Deviance 4.9527, df 10; housing 24 patterns, df 40, Pearson 47.887; apply 220 patterns (sparseness warning).

→ Goodness-of-fit tests (Pearson X² and deviance)

Extra output: Classification table

Concept: Assigns each observation to the category with the highest predicted probability and compares observed with predicted categories. In models with low explanatory power, sparse categories may never be predicted.

Everyday example: If most patients said 'satisfied', the model may predict 'satisfied' for almost everyone and still achieve high accuracy, while catching none of the dissatisfied patients. The table shows this plainly.

What it does: Gives the mean predicted probabilities and the observed × predicted table by the most likely category.

Effect on the result: An extra table.

On the validation data: apply 57.75% correct (the top category is never predicted), mean probabilities .5498 / .3498 / .1004; housing 48.84%.

→ Classification table (confusion matrix)

Partial PO variables

Concept: Lets predictors that violate the proportional odds assumption take a separate slope at each threshold while keeping a common slope for the others. The violation is thus handled locally, without moving to a fully unconstrained model.

Everyday example: If the test of parallel lines flags only waiting time, waiting time gets separate coefficients for the 'dissatisfied / neutral' and 'neutral / satisfied' thresholds, while room type keeps a single coefficient.

What it does: Makes the slopes of the selected predictors threshold-specific (partial proportional odds); a local remedy for a violation of proportional odds. Gives an extra model and an LR against the PO model.

Effect on the result: An extra model. Under separation (very large SE) a warning is written and large numbers are shown in scientific notation.

On the validation data: wine contact: LL -86.2086, AIC 190.417, LR 0.5667 (df 3); apply public: LR 3.8833 (df 1, p .049); all variables = general model (LR 4.0138).

→ Proportional odds (parallel lines) assumption

Overdispersion scale (None (scale = 1), Pearson, Deviance) and Frequency weight

Concept: An overdispersion scale inflates the standard errors by the amount the variability in the data exceeds what the model expects. A frequency weight states that a row represents several identical observations.

Everyday example: If the survey results are stored as summary rows such as 'single room, satisfied: 85 patients', the frequency weight tells the model how many patients each row represents.

What it does: An overdispersion scale for the cumulative logit and a frequency weight for data in count form.

Effect on the result: The scale changes the SEs; the weight changes n and the estimates.

On the validation data: The frequency weight computes the coefficients and n in weighted form; the SEs come from the observed information matrix.

→ Overdispersion and the scale parameter

Example on real data

Data: UCLA ologit (likelihood of applying to graduate school, 400 students)

Variables: target apply (0 = unlikely, 1 = somewhat likely, 2 = very likely), predictors pared, public, gpa

  • Question: Is a student's tendency to apply to graduate school (0 = unlikely, 1 = somewhat likely, 2 = very likely) related to parental education (pared), school type (public) and grade point average (gpa)?
  • Assumption: The proportional odds (parallel lines) assumption is not rejected: χ²(3) = 4.01, p = .260; a single set of coefficients is sufficient.
  • Result: LR χ²(3) = 24.18, p < .001. Students with a parent holding a graduate degree have 2.85 times the odds of being in a higher application category (b = 1.048, 95% CI [1.693, 4.800], p < .001); a 1-point increase in GPA multiplies the odds by 1.85 (b = 0.616, p = .018). The public school effect is not significant (b = -0.059, p = .844). Thresholds 0/1 = 2.203 and 1/2 = 4.299.
  • Effect size: McFadden R² = .033, Nagelkerke R² = .070; explanatory power is low.
  • APA: χ²(3, N = 400) = 24.18, p < .001, McFadden R² = .03; pared OR = 2.85, 95% CI [1.69, 4.80], p < .001.

Related concepts

Likelihood ratio test (LRT)Proportional odds (parallel lines) assumptionLink functionClassification table (confusion matrix)Goodness-of-fit tests (Pearson X² and deviance)Overdispersion and the scale parameterDummy (indicator) variable codingComplete and quasi-complete separation

The numbers in the examples are MerQur's output on independent, publicly available validation data; see the MerQur Technical Note for details.