Encouragement Designs and LATE

By the end of this lesson, you should be able to: tell intention to treat apart from a local average treatment effect and say which question each answers, scale one into the other, name the four compliance types and which one your estimate is about, and explain why the two more direct comparisons are the ones to refuse.

You cannot make anyone use a feature

Alder Stream built a Discover tab. Nobody can be forced to open it, so there's no experiment that assigns usage.

What can be randomised is an email about it. That's an encouragement design, and it's a real experiment: 800,000 users, half emailed at random.

Two thirds of the people who got the email ignored it. Twelve percent of the people who didn't get it opened Discover anyway. Both facts matter.

The number randomisation gives you for free

Compare days active across the two arms, exactly as assigned:

That's the effect of sending the email, averaged over everyone sent one, most of whom ignored it. It needs no assumption beyond the randomisation working. Nobody is dropped, nobody is reclassified, and the comparison is between two groups built by a coin flip.

It's also, very often, the number you should report. If the decision on the table is "do we send this email to everyone", ITT is the answer to that decision, because sending the email to everyone is exactly what was tested.