Statistics Glossary → Firth correction (penalized likelihood)
Firth correction (penalized likelihood)concept
An estimation method for logistic regression that adds a small penalty to the likelihood to reduce small-sample bias and to give finite coefficients even under complete separation.
In logistic regression, if a predictor or a combination of predictors separates the outcome perfectly (for example, everyone above a certain value is ill and everyone below is healthy), the maximum likelihood estimate goes to infinity; the software reports huge coefficients and standard errors or a convergence warning. This is called separation and is common with small samples or rare events. Firth's method adds to the log-likelihood a penalty corresponding to the Jeffreys prior (half the log of the determinant of the information matrix), which shrinks the coefficients slightly towards zero and keeps them finite.
Besides solving the separation problem, Firth's correction also reduces the small-sample bias of maximum likelihood, which tends to overstate coefficients. Profile penalized-likelihood intervals are recommended over Wald intervals. A known side effect is that predicted probabilities are pulled slightly towards 0.5 on average; when probability prediction matters, versions with an extra intercept correction can be used.
An everyday example
In a small study of a rare disease, 6 of 40 people are ill, all 6 carry a particular gene and nobody without the gene is ill. Ordinary logistic regression tries to estimate an infinitely large gene effect and gives a meaningless result. With Firth's correction, the odds ratio is estimated as a large but finite value and an interpretable confidence interval is obtained.
In MerQur
In Logistic Regression, the 'Firth correction' option applies penalized likelihood with the Jeffreys prior and gives finite estimates under separation.
Analyses where this concept appears
Related concepts
Maximum likelihood (ML) estimationComplete and quasi-complete separation