Why your buyer-randomized feed test is biased upward
InterferenceHardProblem. The feed team runs a standard 50/50 buyer-randomized A/B on a new ranking model. Treatment buyers see the new feed, control sees the old one. The test reports GMV +7.5%. A colleague says this is a clean experiment: randomization was correct, the sample is huge, and there is no sample ratio mismatch. Is the number right?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.