New ranker: engagement up, ad revenue and latency down — ship?

Launch or notHard

Problem. An experiment on a new ranking model shows successful-session rate +1.2pp (significant), but search ad revenue −2% and P95 latency +40ms. The PM wants to ship. What's your call?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Define success and ship criteria for a new Copilot answer boxA Windows default change lifted usage — did it cause durable growth? →