How many users (or markets) do you need to detect a 1pp lift?

Power & sample sizeMedium

Problem. Successful-session rate per app-open is ~59%. You want to detect a 1 percentage-point lift. How many users do you need per arm — and why might the real constraint be markets, not users?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Qualified play-through is a ratio metric — analyze it correctlyThe new nav looks worse on day one — when do you call it? →