CCAR-F practice › Context Management & Reliability · Question 101 of 105

Context Management & Reliability

An extraction model outputs a confidence score for each field, and fields below 0.8 go to human review. An audit finds that due_date values reported at 0.9 confidence are wrong about 20% of the time, while vendor_name at 0.9 is almost always right. What is the most effective fix?

Show answer and explanation

Answer: D — Calibrate per-field review thresholds against a labeled validation set

Raw confidence doesn't mean the same thing for every field. Measuring actual accuracy at each confidence level on labeled data, per field, gives thresholds you can rely on. (A) over-reviews reliable fields and still guesses. (B) doesn't calibrate anything. (C) hides the field where errors concentrate.