Product Analytics
Define a metric properly, then use it to work out what moved and why, with examples from real companies' published numbers.
21 lessons in 5 modules, all running on one product's data and building on each other. Start at the top.
0 of 21 done
Module 1 · Foundations
The common metrics every product runs on, how to define one, which one to pick, and what its distribution is hiding.
- 1The Common Metrics Every Product Runs OnA general metrics framework that works for any product: active users and stickiness, a North Star that fits the type of business, then the user journey from acquisition to revenue, with the numbers real companies actually report.
- 2Define the Metric Before You Move ItMeta, Snap, Reddit, Roblox and Duolingo all report daily active users, and they all mean something different by it. What a metric definition has to pin down, and how to choose one that fits the decision.
- 3Which Metric Should This Business Care AboutStart from how the business makes money, find the user action that creates that money, and test the result. A step by step way to find a business's North Star, with the numbers real companies actually report.
- 4The Growth EquationOne equation ties acquisition, retention, churn and resurrection together. Learn each part, see how Duolingo used it to grow daily users 4x, and play with it to see why retention sets the ceiling on growth.
- 5Distributions: Why User Data LiesOn a typical day, only about 1 in 22 Playtika players spends any money. Why user data is almost always lopsided, why the average can describe almost nobody, and which summary answers which question.
Module 2 · The Growth Engine
The five states a user moves through, and the rate that governs each one.
- 6Acquisition: Where Users Come FromWhat acquisition is, why it matters, the channels users come through, and the metrics that tell a good channel from an expensive one. With what Airbnb learned when it paused paid marketing, and what Google pays to be the default search engine.
- 7Activation: The First ExperienceActivation is the moment a new user first gets real value from the product. Why it predicts who stays, how Slack found its threshold of 2,000 messages, and a step by step way to find and improve your own.
- 8Retention CurvesRetention asks how many of the people who started are still here. Learn the three standard ways to define it, how to read the shape of a retention curve, how to read a cohort table, and the mistake that makes recent cohorts look worse than they are.
- 9Churn: Defining It Before Predicting ItChurn is the users you lost. For a subscription, that's easy to see. For most apps, nobody tells you they've left. How to define churn, pick an inactivity threshold, turn monthly churn into yearly, and measure churn in dollars.
- 10Resurrection and Win-BackResurrected users are people who left and came back. In a mature product, the pool of past users keeps growing, so this quiet part of the growth equation can get big. How to define it, measure it, and tell whether a win-back campaign really worked.
Module 3 · Decomposition
Splitting a number until the piece that moved is the only one left.
- 11Funnel Analysis and Drop-offA funnel follows users through the steps toward a goal and shows where they drop off. How to read step and overall conversion, which step to fix, and how to split a change in conversion exactly across steps.
- 12Additive and Multiplicative DecompositionMost metrics are either a sum of parts or a product of drivers. How to split a change in each kind, why the obvious split leaves a leftover term, and how logs split a product exactly. Worked on Reddit, Uber and Meta's reported numbers.
- 13Mix Shift and Simpson's ParadoxA blended average can move even when no group inside it changes, just because the groups changed size. How mix shift works, how to split a change into what happened within groups and what happened to the mix, and when it flips the answer entirely.
- 14Segmentation: Choosing the CutThe same number can be split by country, platform, plan, channel or a dozen other dimensions, and each split tells a different story. How to pick the cut from what could have changed, why slicing everything always finds something, and what makes a cut worth running.
Module 4 · Investigation
Working out what a number should have been before deciding that it fell.
- 15Forecasting and BaselinesTo say a number fell, you need to know what it should have been. How to build a baseline that respects the weekly cycle, score it honestly, and put a range around it, worked on 63 days of real Wikipedia traffic.
- 16Sizing the OpportunityA finding becomes a decision once it has a number on it. How to size a problem as volume times gap times value, why the gap has to be measured rather than assumed, and why the size of a problem isn't the same as what you can win back. Worked on Google's speed experiment and Netflix's password sharing.
- 17Data Quality: Doubt the Instrument FirstA number can change because users changed, or because the way you measure them changed. How to tell the two apart, the checks that catch a broken pipeline even when nothing looks wrong, and why the worst errors make metrics look better, not worse. Worked on Bing's click tracking and Twitter's three-year user overcount.
- 18Why Did It Move?One procedure for every 'why did this metric change' question, run in the same order each time: is it real, is the data right, which part of the equation moved, which cut shows it, what's the size, what do we do. Worked on Netflix's subscriber loss in early 2022.
Module 5 · Decision
Turning the analysis into money, credit and a recommendation you can defend.
- 19Attribution ModelsA customer saw three ads before buying. Which one gets the credit? How the standard attribution models split it, why they all measure which channels were present rather than which ones made a difference, and what eBay and Facebook found when they tested attribution against experiments.
- 20Monetization: LTV, Payback and the Formula That LiesRevenue breaks into users, the share who pay, and what each payer brings in. How to estimate what a user is worth over time, why the textbook LTV formula gets it wrong for most products, and why payback period is the number that decides spending. Worked on Spotify and Snowflake.
- 21Growth Loops and ViralityA funnel ends. A loop feeds itself: new users bring in more new users. How to measure a loop with the viral coefficient, why a coefficient well below 1 is still valuable, why cycle time matters as much, and what Zoom and Dropbox's filings say about loops built into the product.