GMV is heavy-tailed and one coin auction moved the p-value
Test selectionMediumProblem. A feed test reports GMV per buyer up 4.1%, p = 0.03. An analyst notices that one buyer in treatment won a $22,000 coin auction. Removing that single buyer takes the effect to +1.2%, p = 0.41. What do you do?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.