How many accounts do you need for a 0.5pp lift?

Power & sample sizeMedium

Problem. You're A/B testing a checkout change expected to lift payment-success by about 0.5 percentage points off an 80% base. The PM asks: how many accounts do you need, and how long will it run?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

Interview tips

Strong-answer signal: pinning the MDE to business value, tying MDE → sample → runtime, powering on the account unit, and padding for the weekly cycle. Common trap: powering on profiles, picking an MDE out of the air, or stopping at "4 days of traffic" and missing weekly seasonality.

A/B Test & Causal Inference Questions0 / 220
← Choosing the randomization unit when identity is the productWhich test for a rate vs heavy-tailed revenue? →