Regression Discontinuity
By the end of this lesson, you should be able to: recognise a threshold you can exploit, fit and read a discontinuity, treat bandwidth as the judgement call it is, and run the one check that catches the failure mode that matters.
Somebody already randomised something for you
Alder Cloud moves accounts to premium support at 50 seats. Below that, standard support.
An account with 49 seats and an account with 51 seats are the same kind of company. Same size, same maturity, same budget. One of them gets a dedicated support engineer because of where a line was drawn in a pricing meeting.
Nobody designed that as an experiment, and near the line it behaves like one.
The comparison that looks obvious
| Accounts | Retention | |
|---|---|---|
| 50+ seats, premium support | 22,818 | 77.0% |
| Under 50 seats | 37,182 | 62.0% |
| Gap | +24.23% |
The true effect is +3.90%. The gap is 6.2 times too big, and the reason is visible the moment you look at who's in each group:
| Mean seats | |
|---|---|
| Above the line | 104 |
| Below the line | 26 |
That's a comparison between 104-seat companies and 26-seat companies. Retention rises with size for a dozen reasons that have nothing to do with support tiers. The comparison measures company size and calls it support.