40 metrics across 12 segments — how many wins are fake?
Multiple testingMediumProblem. A teammate reports that their test was flat overall, but significant for the Enterprise segment on mobile. They checked 40 metrics across 12 segments and found 3 significant results. They want to ship to that segment. What do you say?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.