Sensitivity Analysis and Choosing a Method

By the end of this lesson, you should be able to: put a number on how strong an unmeasured confounder would have to be, compare that number with confounders you did measure, say what a sensitivity analysis does and doesn't prove, and choose a method based on what the design can support.

How strong would a hidden confounder have to be?

Every method in Modules 2 to 5 rests on an assumption the data can't check: no unmeasured confounding, parallel trends, exclusion. Sensitivity analysis replaces "we assume there's no unmeasured confounder" with a number: how strong one would have to be to change the conclusion.

The idea goes back to the 1950s smoking debate. A study by Hammond and Horn found smokers had 10.7 times the risk of lung cancer of non-smokers. The statistician R. A. Fisher suggested a gene might make people both more likely to smoke and more likely to get cancer. In 1959, Cornfield and colleagues answered with arithmetic: for such a factor to explain the whole association, it would have to be at least as strongly linked to smoking and to cancer as smoking was to cancer. Nobody had found a factor anywhere near that strong.

The E-value

VanderWeele and Ding turned this into a single number, the E-value: the minimum strength of association, as a risk ratio, that an unmeasured confounder would need with both the treatment and the outcome, beyond the measured covariates, to explain an estimate away.

For a risk ratio $RR$ above 1:

\text{E-value} = RR + \sqrt{RR \times (RR - 1)}

For a protective effect, with $RR$ below 1, use $1/RR$ in its place.

For smoking, the E-value is about 21. A hidden factor would need a risk ratio of about 21 with smoking and with lung cancer to explain the result away. VanderWeele and Ding also recommend computing the E-value for the end of the confidence interval nearest 1. For smoking's lower limit of 8.0, it's 15.5.

Try it: how strong would a hidden confounder need to be?

Observed risk ratio, after adjustment10.7Confounder's link to treatment and to outcome10.7×

The lowest the true risk ratio could be if an unmeasured confounder this strong existed. Where the line drops to 1, the confounder could explain the whole effect away.

E-value

20.89

strength needed to explain it away

Bias from this confounder

÷ 5.61

at most

Lowest true risk ratio

1.91

still above 1

With the defaults, the observed risk ratio is 10.7 and the confounder is linked to both by 10.7, the strength Cornfield's argument considered. The true risk ratio could be no lower than 1.91. Even that strong a confounder couldn't explain the effect away. But it could shrink it from 10.7 to under 2.