MerQuR+Statistics Glossary

Statistics Glossary → Sensitivity analysis

Sensitivity analysisconcept

An approach that checks how robust the result of an analysis is to reasonable changes in the data, the assumptions or the methodological choices.

Every analysis rests on a set of choices: which observations are included, how missing data are handled, which prior or model is used, and so on. Sensitivity analysis varies these choices systematically and checks whether the main result (the direction, size and significance of the effect) changes. In meta-analysis a common form is leave-one-out analysis: each study is removed in turn and the pooled effect is recomputed. In Bayesian analysis results are compared across different priors, and in missing-data analysis across different assumptions about why data are missing.

If the result holds up across the changes, confidence in the finding grows; if removing a single study, observation or assumption reverses it, that should be reported openly. Sensitivity analysis is not a hypothesis test and does not pick the 'right' model; its purpose is to make visible which choices the conclusion depends on. The scenarios to be tried should be planned in advance, since trying options until the result looks good introduces bias.

An everyday example

A family is working out whether they can afford a mortgage. The numbers look comfortable if the interest rate stays at 3%. If they redo the calculation at 4% and 5% and the monthly payment still stays below a third of their income, the decision is robust; if the budget is already in deficit at 4%, the decision hangs on a single optimistic assumption.

In MerQur

In Meta-Analysis, the 'Sensitivity: Leave-one-out' option re-estimates the model with each study left out in turn, and 'Sensitivity: Influence diagnostics' flags influential studies. In Multiple Imputation, the 'MNAR sensitivity variable and MNAR δ shifts' show how the results change under a missing-not-at-random assumption; in Bayesian Linear Regression, the 'Prior sensitivity' extra output gives the posterior means and log marginal likelihood over a grid of τ values.

Analyses where this concept appears

Meta-AnalysisBayesian Linear RegressionMultiple Imputation

Related concepts

Influence diagnostics (Cook's distance, leverage, DFBETAS)Prior distribution