The model wins offline but loses online — which do you trust?

Decide from resultsMedium

Problem. A new recommendation model beat the current one on offline metrics (precision, recall, ranking loss), but in the live A/B its watch time is flat-to-negative. Which result do you trust, and why would they disagree?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Two experiments overlap on the same surface — are they interfering?The watch-time lift is fading over the days-in window — real or novelty? →