Surrogate Metrics & the OEC

By the end of this article, you should be able to: choose a single decision metric before a test starts, validate an early proxy for a delayed outcome, and catch the trap that makes a surrogate lie to you.

Step 1: Meet the problem

A fitness app redesigns its onboarding flow. What the team actually cares about is whether people are still active in 12 months, but nobody can wait a year to decide whether to keep the new flow. Recent users haven't even had 12 months yet, their long-term outcome is literally not observable, and the team needs a ship decision in a couple of weeks.

Two separate problems are tangled together here: which single number will you use to decide, and what do you look at in the next two weeks that actually tells you something true about month 12?

Step 2: Pick one number to decide by

The first problem has a name: the OEC, or Overall Evaluation Criterion. It's the one metric, or a small weighted combination, that the team agrees on before the test starts. Deciding this up front matters because if you wait until after the results come in, it's tempting to scan twenty metrics and declare victory on whichever one moved, which isn't a real result, it's cherry-picking dressed up as one.