Testing a new guideline — what's the randomization unit?
Experiment designHardProblem. You want to A/B test a new labeling guideline against the old one. Design the experiment — in particular, what do you randomize (items, annotators, or tasks), and what breaks if you choose wrong?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.