CCAO-F practice › Output Evaluation and Validation · Question 17 of 100

Output Evaluation and Validation

A consultant receives a market-sizing estimate from Claude with several precise figures. She asks, 'Are you confident these numbers are accurate?' and Claude replies that it is highly confident. What should she do?

Show answer and explanation

Answer: D — Trace each figure to a source she can check, and flag or drop the rest

A model's stated confidence is not a reliable signal of accuracy. The figures need to be traced to sources she can check (d). Treating the model's confirmation as verification is the core mistake (a). Confidence scores from the model have the same problem in numeric form (b). Getting the same answer twice shows consistency, not truth (c).