Synthetic Control

By the end of this lesson, you should be able to: explain why one treated unit defeats difference-in-differences, state the two constraints that define synthetic control and what each one prevents, read a pre-period fit honestly, and know why the best-fitting model here is the worst estimator.

One city, and nothing to compare it to

Alder Go launched a loyalty programme in Bristol. Thirty other cities did not get it, so the obvious move is difference-in-differences from lesson 10: compare Bristol's change against the other cities' change.

Nearly double. And this time the failure isn't a trend that was already there, it's something more basic.

DiD uses the average of the 30 donor cities as its counterfactual. That is only the right counterfactual if Bristol behaves like the average city, and Bristol does not. It's coastal, it's mid-sized, its weekend pattern is unusual. Averaging thirty cities produces a city that doesn't exist and that Bristol was never going to resemble.

Build the comparison instead of assuming it

Synthetic control asks a better question. Rather than is Bristol like the average, it asks: is there some weighted combination of these cities that behaves like Bristol?

Pick weights $w_j$ for each donor city so that the weighted blend matches Bristol as closely as possible before the launch:

$\min_{w} \sum_{t < T_0} \left( Y_{\text{Bristol},t} - \sum_j w_j Y_{j,t} \right)^2$

subject to two constraints that are the entire method:

$w_j \ge 0 \quad \text{for every } j, \qquad \sum_j w_j = 1$

Solve it on Bristol's 26 pre-launch weeks and you get:

Five cities out of thirty, and the rest get exactly nothing. That sparsity is not something anyone asked for; it falls out of the constraints.

How well does the blend track Bristol? Over 26 pre-launch weeks, the root mean squared error is 33 orders a week against a mean near 10,000, or 0.33%. For half a year, that combination of five cities is Bristol to within a third of a percent.

Look at the left panel. The grey line is the donor average, which is what DiD uses: a different level, a different slope, not Bristol. The dashed line is the synthetic blend, which sits on top of Bristol until the launch and separates afterwards.