Statistics Glossary → Conjugate prior (normal-inverse-gamma, Zellner's g)
Conjugate prior (normal-inverse-gamma, Zellner's g)concept
A prior distribution that, combined with the likelihood, yields a posterior from the same family as the prior, so that the posterior can be computed in closed form.
For linear regression with normal errors, the conjugate prior is the normal-inverse-gamma (NIG) distribution: given σ², the coefficients are normal, and σ² follows an inverse gamma distribution. Combined with the data, the posterior is also NIG, and the posterior mean is an average of the prior mean and the least-squares estimate weighted by their precisions. This gives fast, exact results without simulation (MCMC). Conjugacy exists in other models too: the beta prior for the binomial and the gamma prior for the Poisson.
Zellner's g prior is a special form of the NIG: the prior covariance of the coefficients is taken as g·σ²(X'X)⁻¹, so a single value g sets the strength of the prior. With a prior centred at zero, the posterior mean shrinks the least-squares estimate by a factor of g / (1 + g); g = n is a common 'unit information' choice. Conjugate priors are chosen for mathematical convenience; whether the prior reflects genuinely held beliefs, and how sensitive the result is to the choice of prior, should be assessed separately.
An everyday example
A company estimates the effect of advertising spend on sales. From past campaigns it knows the effect is around 2 and enters this with an NIG prior. The new data alone suggest 3; because data are plentiful, the posterior mean is 2.8, whereas with little data it would sit closer to the prior. With a Zellner prior and g = 50, the zero-centred prior would shrink the least-squares estimate by 50/51, that is, only slightly.
In MerQur
The 'Prior type (Conjugate (NIG) / Reference (Jeffreys) / Zellner g-prior) and g' option in Bayesian Linear Regression; g takes effect only with the g-prior, and entering g = 0 sets g = n.