CCAO-F practice › Output Evaluation and Validation · Question 17 of 100
Output Evaluation and Validation
A consultant receives a market-sizing estimate from Claude with several precise figures. She asks, 'Are you confident these numbers are accurate?' and Claude replies that it is highly confident. What should she do?
Show answer and explanation
Answer: D — Trace each figure to a source she can check, and flag or drop the rest
A model's stated confidence is not a reliable signal of accuracy. The figures need to be traced to sources she can check (d). Treating the model's confirmation as verification is the core mistake (a). Confidence scores from the model have the same problem in numeric form (b). Getting the same answer twice shows consistency, not truth (c).
Keep going
More Output Evaluation and Validation questions
- An operations analyst has 15 minutes to review Claude's summary of a supplier's 30-page…
- A teacher uses Claude for several tasks this week. Which output most warrants checking…
- A grant writer asks Claude, with no files uploaded and no search enabled, for a…
- A program manager reviews a Claude-drafted board update. The executive summary says all…
- A recruiter's prompt asked Claude to write an inclusive job ad. The draft describes the…