Your Feed scorecard has 60 metrics — how many 'wins' are fake?
Multiple testingHardProblem. Every Feed experiment reports a scorecard of ~60 metrics. This experiment shows 3 metrics "significant" at p < 0.05, the rest flat. The team is excited about the 3 wins. Should they be? How do you read a big scorecard honestly?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.
Strong-answer signal: computing the expected false positives (m × α), insisting on a pre-registered primary, applying an FDR correction, and demanding a coherent metric story. Common trap: celebrating cherry-picked significant metrics from a large scorecard with no multiplicity correction.