The encoder helps on average and hurts the players you are losing

Heterogeneous effectsHard

Problem. The encoder test is done. Flawless session rate is up 1.0 point overall and it is clearly significant. But when you split by the player's bandwidth, the low-bandwidth segment got worse. Product wants to ship. What do you say?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Your metric is p99 latency. You cannot t-test a quantileEstimate the data center's effect with no randomization →