MerQuR+Statistics Glossary

Statistics Glossary → Non-informative (flat, Jeffreys) prior

Non-informative (flat, Jeffreys) priorconcept

A prior distribution in Bayesian analysis that carries no strong prior belief about a parameter and aims to let the results be determined mainly by the data.

A flat (uniform) prior gives equal weight to all values of the parameter: p(θ) ∝ 1. Defined over all real numbers, its integral is infinite, so it is an 'improper' prior; even so, if the likelihood carries enough information it yields a proper posterior (one that integrates to 1). Under a flat prior the posterior is proportional to the likelihood, so the posterior mode equals the maximum likelihood estimate. The Jeffreys prior is proportional to the square root of the Fisher information and is invariant to reparameterization (e.g. defining it via the standard deviation or the variance gives the same result); for scale parameters it takes the form p(σ) ∝ 1/σ.

A 'non-informative' prior is never entirely uninformative: a prior that is flat on one scale is not flat on a transformed scale. Improper priors cannot be used in computing Bayes factors, because their constants are undefined; Bayesian hypothesis tests (e.g. the JZS prior) therefore use proper, weakly informative priors such as the Cauchy for the parameter being tested, while parameters common to both models, such as the intercept and variance, can be given improper priors. In logistic regression with complete separation, a flat prior can lead to infinite coefficients; the Jeffreys prior (equivalent to Firth's correction) gives finite estimates in that case.

An everyday example

A dice factory wants to estimate the probability that a new die shows a 6, with no prior information. With a flat prior, all probabilities between 0 and 1 start out as equally plausible. After 12 sixes in 60 throws, the posterior is determined almost entirely by these data; the posterior mean is (12 + 1)/(60 + 2) ≈ 0.21, very close to the observed rate of 0.20.

In MerQur

In Bayesian linear regression the reference prior under 'Prior type' (Jeffreys, p(β, σ²) ∝ 1/σ²) makes the posterior mean equal to OLS and the interval equal to the t interval; the 'Flat prior for the intercept' option gives the intercept the improper prior p(β0) ∝ 1 under the conjugate prior. In the Bayesian t-test the descriptives under 'Additional output' give the credible interval of the mean with a Jeffreys prior; the 'Firth correction' in logistic regression applies a penalized likelihood with the Jeffreys prior and gives finite estimates under separation.

Analyses where this concept appears

Logistic RegressionBayesian t-Test (JZS Bayes Factor)Bayesian Linear Regression

Related concepts

Prior distributionConjugate prior (normal-inverse-gamma, Zellner's g)