Test a new Trust Index model on live traffic

Experiment designHard

Problem. The backtest says a new Trust Index model catches more fraud. Before rolling it out to all customers, the team wants to test it on live traffic for a set of customers who've agreed to take part. Design the test.

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

Fraud that gets through is 0.5% of users. Size a test to detect a 20% drop. →