A new search ranker wins offline — how do you test it without fooling yourself?
Experiment designHardProblem. A new search ranking model beats the current one on offline metrics (booking-based NDCG). You want to validate it online. What are the traps — especially position bias — and how do you design the test?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.