Regression Discontinuity

By the end of this lesson, you should be able to: recognise a threshold you can use, estimate the jump at the threshold, treat bandwidth as the judgement call it is, and run the checks that catch the failure that matters most.

A rule that splits nearly identical cases

Yelp shows each restaurant's average rating rounded to the nearest half star.1 A restaurant with an average of 3.24 shows 3 stars. One with 3.25 shows 3.5 stars.

Those two restaurants are, for practical purposes, equally good. Their reviewers rated them almost exactly the same. But diners searching Yelp see half a star of difference.

That's a regression discontinuity: a rule assigns treatment based on whether a score, called the running variable, crosses a threshold. Units just either side are alike in everything except the treatment, so the jump in the outcome at the threshold measures the treatment's effect.

Thresholds like this are everywhere in products: free-shipping minimums, loyalty tiers, credit score cut-offs, ranking positions, eligibility rules.

Why the simple comparison fails

Does a better Yelp rating bring in more revenue? Michael Luca at Harvard Business School combined Yelp reviews with revenue data from the Washington State Department of Revenue, covering every restaurant in Seattle from 2003 to 2009, about 1,587 open in a typical quarter.1

A regression of revenue on rating found that a one-star increase was associated with 5.4% more revenue, even comparing each restaurant with itself over time. But a restaurant's Yelp rating can rise because its food improved, a newspaper reviewed it, or word of mouth spread. Those raise revenue too, and some aren't measured.

The rounding threshold avoids this. Whether a restaurant lands at 3.24 or 3.25 has nothing to do with newspaper reviews or word of mouth. Luca compared restaurants within 0.1 stars of a rounding threshold. Crossing it, and gaining half a displayed star, raised revenue by about 4.5%: about 9% per star.