Which test for a success rate vs queries-per-session?

Test selectionMedium

Problem. For a Bing experiment you have successful-session rate (a rate), queries-per-session, and revenue-per-session (both skewed). A teammate runs a Student's t-test on all three. Which test for each, and what's the catch with the skewed and the ratio metrics?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← Powering for a 0.2pp success lift (and speeding it up)Success up, ad revenue down, with a novelty curve — ship? →