LLM-as-Judge for Measurement

By the end of this article, you should be able to: say when an LLM labeller is good enough to sit behind a metric, run the golden-set loop that turns a policy into a labeller, name the biases LLM judges are known to have, and decide when to use several judges instead of one.

Why use an LLM as the labeller

Every quality or safety metric built on labels has a cost problem. Careful human labels are slow and expensive, so they get collected rarely, and a metric you can only compute once a quarter can't tell you what happened last week.

LLMs changed that cost. Three findings show how much:

1. Against crowd workers, LLMs are better and far cheaper. On 6,183 tweets and news articles, ChatGPT's accuracy beat crowd workers on Amazon Mechanical Turk by about 25 percentage points on average, at under $0.003 per label, about thirty times cheaper.1
2. Against trained moderators, they're close. Pinterest's LLM labeller came within about 3.2% of human reviewers' accuracy and 0.5% of their F1 score, on test sets labelled by policy experts. It cut labelling time by about 15 times and cost by more than 10 times, which turned a prevalence study run monthly or quarterly into a daily metric.2
3. Against experts, not yet. OpenAI found GPT-4's labels "similar to human moderators with light training", but both were "overperformed by experienced, well-trained human moderators".3

So an LLM is a strong, cheap labeller, not a perfect one. Before an LLM's labels can sit behind a metric, someone has to show how good they are, and keep checking.

Write the policy, then build the golden set

The order matters. A policy says what counts: what's a violation, what isn't, and how to decide the edge cases. A prompt is the policy turned into instructions for the model. Skip the policy and go straight to prompting, and the judgment calls end up hidden inside prompt wording nobody can review.

OpenAI describes the loop it uses to develop content policies:3

1. Policy experts write the policy and label a small set of examples by hand. This is the golden set: items whose correct label you trust.
2. GPT-4 reads the policy and labels the same examples without seeing the answers.
3. Experts study every disagreement. They ask the model to explain its reasoning, find where the policy is ambiguous, and clarify the policy.
4. Repeat steps 2 and 3 until the policy is good enough.

Step 3 is where the value is. Many disagreements aren't model mistakes. They're places where the policy didn't say what to do, and two careful humans would have disagreed too. Fixing the policy makes human reviewers more consistent as well. OpenAI reports this process cut policy development "from months to hours."