What you’ll be able to do
- Design a judge prompt that returns a structured verdict, not a bare score
- Identify position, verbosity, and self-preference bias in a judge’s grading
- Apply the mitigation that matches each bias
- Recognise when a judge should be spot-checked by a human rather than trusted outright
What you need to know
The pattern
Model-graded evaluation, also called LLM-as-judge, uses a model call to grade another model's output against an explicit rubric — the scalable option from lesson 4.2's three grading methods, for criteria too subjective for a deterministic check: does this response actually answer what was asked, does the tone match the brand, is the explanation faithful to the source document rather than just plausible-sounding.
Input: {{question}} Source: {{source_document}} Response to grade: {{candidate_response}}
Return JSON: { “pass”: bool, “failed_criteria”: […], “reason”: ”…” }
A bare 1-10 score with no reasoning is nearly useless — it can’t be audited, and a human reviewing the judge’s work has nothing to check it against. A structured verdict against named criteria, with a reason, can be spot-checked and disagreed with, which is the whole point of keeping a human in the loop at all.
Three biases that skew a judge, and the fix for each
- Position bias — in a pairwise comparison ("which response is better, A or B"), the judge favors whichever position comes first or second more often than chance. Mitigation: run the comparison twice with the order swapped and check for agreement; a judge that flips its verdict when only the order changed isn't measuring quality.
- Verbosity bias — a longer answer gets graded as more thorough regardless of whether the extra length adds anything. Mitigation: include response length as an explicit non-factor in the rubric, or normalize by controlling for length in the comparison set.
- Self-preference bias — a model grading outputs from its own family tends to rate them more favorably than outputs from a different model, independent of actual quality. Mitigation: use a separate model as judge from the one under test where the comparison matters, or calibrate the judge's scores against human-labeled examples first.
The distinction the exam reaches for: these are properties of the grading process, not the response being graded. A high judge score achieved by exploiting one of these isn't a false positive in the response — it's a judge that needs recalibrating.
A judge is not ground truth
Model grading scales past what human review can reach, which is exactly why it needs the same suspicion any scaled, imperfect proxy deserves. Calibrate the judge initially against a human-labeled sample, and keep auditing a small ongoing sample rather than trusting the judge indefinitely once it passed calibration once. A judge that agreed with humans on last quarter's dataset can still drift as the system under test changes in ways the judge was never checked against.
Key concept
A judge model needs the same rubric, input, and output every time, plus a structured verdict with a reason — and it needs its position, verbosity, and self-preference biases checked, not assumed away.
Practice scenario
Work it through, then open this
Fixed ordering is the exact setup that makes position bias invisible — a genuine improvement and an artifact of always appearing in the favored slot produce the same result. The fix isn’t trusting the win rate more, it’s rerunning the same comparisons with the order swapped. If the win rate holds under both orderings, that’s a real signal. If it collapses, the original result was measuring position, not quality.
Build exercise — Stress-test a judge for bias before trusting it
Intermediate · 25 min
What you’ll learn
- Detecting position bias by swapping comparison order
- Writing a rubric that returns an auditable verdict
-
Write a judge prompt with an explicit rubric (named criteria, not “which is better”) that returns structured JSON with a pass/fail per criterion and a reason.
- Why: A bare score can’t be audited. A structured verdict against named criteria can be checked and disputed.
- You should see: Output you could hand to a human reviewer and have them meaningfully agree or disagree with.
-
Run the same pairwise comparison twice, swapping which response is presented first the second time.
- Why: This is the direct test for position bias — a real quality difference should survive the swap.
- You should see: Either consistent verdicts across both orderings, or a flip that tells you the judge is measuring position, not quality.
-
Pull 5-10 of the judge’s verdicts and have a human grade the same examples independently, then compare.
- Why: A judge is a scaled proxy, not ground truth. Periodic human audit is what catches drift before it compounds.
- You should see: Either close agreement, or specific disagreements that tell you what to fix in the rubric.
Exam traps
Asking the judge for a single number with no reasoning
An unstructured score can’t be audited or disputed. Ask for named criteria and a reason.
Always presenting the candidate answers in the same order
That setup makes position bias indistinguishable from a real quality difference. Swap the order and check for agreement.
Assuming a longer answer the judge preferred is actually the better one
Verbosity bias grades length as thoroughness. Control for it explicitly in the rubric.
Treating judge output as ground truth with no human audit
A judge is a scaled proxy calibrated once, not a permanent substitute for human review. Keep auditing a sample.