Design an A/B test for a new model version — what's the OEC?
Experiment designHardProblem. You're asked to A/B test a new ChatGPT model version against the current one. The new model changes quality, tone, latency, and cost all at once. Design the experiment: unit, metrics, and the Overall Evaluation Criterion.
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.