Staggered Rollouts

By the end of this lesson, you should be able to: recognise when a staggered rollout makes two-way fixed effects unsafe, explain which comparisons go wrong and why, and estimate the effect using only comparisons that hold up.

A result nobody should believe

Alder Stream shipped a feature to 24 countries in six waves, six weeks apart. The obvious regression:

$Y_{it} = \alpha_i + \tau_t + \delta D_{it} + \varepsilon_{it}$

Country fixed effects, week fixed effects, a dummy for having the feature. This is the specification in every textbook and most papers.

The feature hurt, significantly.

Every country's effect is positive. The generator planted it that way. There's no country, no week, and no cohort anywhere in this data where the feature does harm.

What the regression is really doing

With one treated unit and two periods, TWFE is exactly the 2×2 from lesson 10. With staggered timing it becomes something else: a weighted average of every 2×2 comparison available in the data.1

Some of those comparisons are fine. Some are not, and the difference is which group plays the control.

Comparison A: a later cohort against countries not yet treated. Wave 4 adopts at week 26. Waves 5 and 6 haven't adopted yet, so they're clean controls. This is a proper DiD.

Comparison B: a later cohort against countries already treated. Wave 6 adopts at week 38. Wave 1 adopted at week 8 and is sitting right there in the data, so the regression uses it as a control.

Comparison B is the problem, and only under one condition: when effects change over time.

Wave 1 has had the feature for 30 weeks by the time wave 6 gets it. If the effect grows with exposure, wave 1 is still improving during wave 6's post period. Using it as a control means subtracting a rising series from a rising series, and what's left can easily be negative.

Split the comparisons and measure them:

There it is. The valid comparisons say the feature works. The invalid ones say it hurts.