A team building a contract-review assistant has an evaluation set of 200 questions written by its prompt engineers. Scores have climbed steadily over six weeks of prompt iteration, but in a pilot, in-house counsel find the assistant struggles with scanned exhibits, non-English clauses and questions the contract doesn't address. What is the best first improvement to the evaluation dataset?

Answer: A — Add sampled pilot queries and the edge cases the pilot exposed, plus a held-out split not used for tuning

(a) is right: the set reflects what its authors expected and has been tuned against, so it needs real traffic, the specific edge cases the pilot exposed, and a held-out split so ship decisions aren't made on data the prompt was fitted to. (b) multiplies the existing blind spots. (c) improves the sources but discards regression coverage and treats the dataset as a one-off rather than versioned and maintained. (d) changes grading, not coverage; a better judge can't score cases that aren't in the set.