MerQuR+Statistics Glossary

Statistics Glossary → Observed (post hoc) power

Observed (post hoc) powerconcept

The statistical power of a test computed by treating the effect size observed in the sample as if it were the true effect.

Statistical power is the probability that a test detects a real effect, and it is normally computed before a study from the effect size one expects. Observed power is computed after the analysis by plugging in the observed F or effect size as if it were the population value. In ANOVA the noncentrality parameter of the noncentral F distribution is taken as λ = F·df₁, and power is the probability that F exceeds its critical value.

Observed power is a one-to-one transformation of the p value: the larger p is, the lower the observed power, and at exactly p = 0.05 it is about 50%. Saying 'power was low, so the effect was not found' after a non-significant result therefore only restates that p was large (Hoenig and Heisey, 2001). To describe how sensitive a study was, an a priori power analysis or a confidence interval for the effect size is far more informative.

An everyday example

A teacher compares two teaching methods with 12 students each and finds p = 0.30. The software reports an observed power of 18%. This number adds nothing new; it is simply p = 0.30 expressed another way. What helps the teacher more is to look at how wide the confidence interval for the difference is.

In MerQur

In two-way ANOVA, the 'Extra output: Observed power' checkbox reports the power of each effect computed from λ = F·df1; the output notes that this value is a transformation of the observed p and has limited interpretive value.

Analyses where this concept appears

Two-Way ANOVA

Related concepts

Type I and Type II errorsEffect size