Sensitivity Analysis and Choosing a Method

By the end of this lesson, you should be able to: put a number on how strong an unmeasured confounder would have to be, benchmark that number against covariates you did measure, say what a sensitivity bound does and does not certify, and triage a real readout to the method its design can actually support.

Part A: how strong would it have to be?

Every estimate in modules 2 through 5 rests on something the data can't check. Sensitivity analysis replaces "we assume no unmeasured confounding" with a number.

Take the estimate this track couldn't finish. Lesson 7 weighted Alder Cloud's teams and got +14.88% against a planted truth of +5.83%, both in logs. Sensitivity analysis works on the ratio scale, where those are 1.1604 and 1.0600.

The E-value is the smallest association a confounder would need with both treatment and outcome, over and above everything already adjusted for, to explain an estimate away entirely.1

$\text{E-value} = RR + \sqrt{RR \times (RR - 1)} = 1.59$

As a sentence: something would have to make a team 1.59x more likely to adopt and 1.59x more productive, beyond what the four covariates capture, before this estimate could be nothing at all.

The yardstick that makes it mean something

On its own, 1.59 is an abstraction. It gets useful when you compare it against covariates you did measure, because those are things the team has intuition about.2

The E-value to beat is 1.59, and two of the four covariates exceed it on their own. So the honest reading is: a confounder no stronger than something already sitting in the table would be enough. That's a fragile estimate, and you can say so without ever naming the confounder.