Label-Error Correction — Rogan–Gladen

By the end of this article, you should be able to: explain why an imperfect labeler biases a prevalence estimate, show why rare harms are especially fragile, apply the Rogan–Gladen correction, and know when you don't need it.

Step 1: Meet the problem

You're measuring how common a rare harm is by labeling a sample. Your labeler — an LLM, a classifier, or even a human — is good but not perfect. It misses some real violations and false-flags some safe content. The natural instinct is to trust the labeled rate as the answer. For a rare harm, that instinct is badly wrong, and the reason is worth seeing concretely.

Step 2: Name the two error rates

Any imperfect binary labeler has two error rates, borrowed from diagnostic testing:

• Sensitivity $r = P(\text{flag} \mid \text{truly violating})$ — of real violations, the share the labeler catches (its true-positive rate).
• False-positive rate $f = P(\text{flag} \mid \text{truly safe})$ — of safe items, the share it wrongly flags. (Specificity is $1 - f$.)

A "95% accurate" labeler can have very different $r$ and $f$, and for rare-harm prevalence, $f$ is the one that hurts.