Concept Crash Course

The standalone explainers. One concept per article, reachable from any practice answer that leans on it, in no particular order.

6 reference guides. They stand on their own, so read whichever one you need.

0 of 6 done
Start →
  1. 1Surrogate Metrics & the OECDeciding on a long-term outcome you can't wait for, by measuring an early signal that stands in for it. How Netflix checked a two-week surrogate against two-month results in 200 A/B tests, and how a heart drug that fixed its surrogate killed more patients.
  2. 2Customer Lifetime Value (LTV)How much a customer is worth over the whole relationship, and why the textbook shortcut gets it wrong. Why churn falls over time even when no customer changes, and how a simple model projected 12 years of real subscription retention from 7 years of data, within a few percent.
  3. 3Price Elasticity of DemandHow much demand moves when the price changes, and when a price rise raises revenue. How Uber measured elasticities of about −0.5 from the rounding in its surge prices, and why Netflix's 2019 US price rise can't be read from a before-and-after.
  4. 4Prevalence Estimation (Design-Based Sampling)How platforms measure what share of views went to harmful content, when the harm is rare and every label is expensive. Meta's and YouTube's published prevalence metrics, and how Pinterest samples where the violations are and still gets an unbiased number.
  5. 5Label-Error Correction (Rogan–Gladen)An imperfect labeller biases a prevalence estimate, and when the thing you're counting is rare, a tiny false-positive rate can double it. The one-line correction from 1978 epidemiology, how it decided a famous COVID antibody study, and how Pinterest decides when to use it.
  6. 6LLM-as-Judge for MeasurementUsing a large language model as the labeller behind a metric, and the checks that make its labels trustworthy. How OpenAI and Pinterest run the golden-set loop and launch gate, the biases LLM judges are known to have, and when a panel of judges beats one.