Annotation agreement dropped on a project — what happened?
Diagnose a problemMediumProblem. On an ongoing annotation project, inter-annotator agreement fell from 0.82 to 0.68 over a few weeks. The customer is worried about data quality. What happened, and what do you do?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.