Size a test whose win is a rare, delayed Premium conversion

Power & sample sizeMedium

Problem. You are testing a change meant to lift free-to-Premium conversion. Conversion is rare (about 2% of eligible members) and shows up over weeks. How many members do you need, and how long do you run it?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Your invitation feature has network spillover — is the A/B valid?Which test for connections-per-member and InMail response rate? →