Recs get more starts but lower completion — do you ship?

Metric designHard

Problem. A recommendation change raises title starts +8% but completion rate and capped watch time fall. The team wants to ship on the starts lift. What's your call?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Design an A/B test for a ranking change — what's the OEC?The model wins offline but loses online — which do you trust? →