Size a test for a 1 point lift in flawless session rate

Power & sample sizeMedium

Problem. Flawless session rate sits around 90.8%. Engineering thinks the new encoder can add about 1 point. How many players do you need, and for how long?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Randomize a new encoder by session or by player?Your metric is p99 latency. You cannot t-test a quantile →