A messaging feature spreads person-to-person — how do you randomize?

Experiment designHard

Problem. You're testing a new message feature (say, a richer reply/reaction). If you randomize users individually, the result will be biased, because a treated user's messages reach untreated partners. How do you design the experiment?

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

← You can't read messages — how do you pick a metric to experiment on?Retention is already 92% — can your test even detect a change? →