Difference-in-Differences
By the end of this lesson, you should be able to: estimate an effect with difference-in-differences, explain what it removes and what it assumes, use a second comparison to check it, and say how to handle uncertainty when only one group was treated.
The idea
Lesson 9 showed that a before-and-after comparison credits everything that changed over time to the treatment. Difference-in-differences (DiD) fixes that by finding a comparison group that went through the same period without the treatment.
- Measure how much the treated group changed, before to after.
- Measure how much the comparison group changed over the same period.
- Subtract the second from the first.
The comparison group's change stands in for what would have happened to the treated group without the treatment: the season, the economy, the news, everything that hit both. Subtracting it leaves the treatment's effect.
Notice that the two groups don't have to start at the same level. DiD compares changes, not levels. If the treated group is always 20% bigger, that difference cancels out.
The classic example: New Jersey's minimum wage
On 1 April 1992, New Jersey raised its minimum wage from $4.25 to $5.05 an hour. Pennsylvania, next door, didn't. Economists David Card and Alan Krueger surveyed 410 fast-food restaurants in New Jersey and eastern Pennsylvania before and after the increase.
Before the increase, Pennsylvania stores averaged 23.3 full-time-equivalent workers, and New Jersey stores 20.4. The levels were different, which DiD allows. What matters is how each changed. Between the two surveys, employment at Pennsylvania stores fell, while New Jersey stores gained a little. The difference-in-differences was +2.76 workers per store, about 13%, in New Jersey's favour.
Many economists had expected a higher minimum wage to reduce employment. This study found no sign of that, and it was widely debated. Other researchers, using payroll records from a different set of restaurants, reached the opposite conclusion. The method was the same. The data was different, which is a reminder that DiD is only as good as the measurements it subtracts.