Design an A/B test for a checkout change — what's the OEC?

Experiment designMedium

Problem. The team ships a streamlined checkout to lift order conversion. Design the A/B test — unit, metrics, and the Overall Evaluation Criterion.

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

Interview tips

Strong-answer signal: an orders+retention OEC (not checkout conversion), the right guardrails including marketplace supply, and CUPED for power. Common trap: optimizing checkout conversion alone, ignoring cancellations, retention, and supply strain.

A/B Test & Causal Inference Questions0 / 220
You can't A/B a marketing channel — measure its incrementality →