Fraud that gets through is 0.5% of users. Size a test to detect a 20% drop.
Power & sample sizeMediumProblem. A new rule is expected to cut fraud that gets through at signup from about 0.5% of users to about 0.4%. The participating customers score about 25,000 signups a day in total. The PM asks how long the test needs to run. How do you answer?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.