CCAR-F practice › Context Management & Reliability · Question 100 of 105
Context Management & Reliability
An invoice-extraction pipeline shows 97% field accuracy overall, and the team plans to remove human review entirely. Inputs include standard invoices, credit notes, and handwritten receipts. What is the best first step?
Show answer and explanation
Answer: B — Measure accuracy by document type and field, and confirm each segment meets the bar
An aggregate can mask poor performance on specific document types or fields. Validate by segment before reducing review. (A) trusts the aggregate. (C) uses uncalibrated confidence and still skips segment validation. (D) makes the aggregate more precise without revealing weak segments.
Keep going
More Context Management & Reliability questions
- An extraction model outputs a confidence score for each field, and fields below 0.8 go…
- A team has automated acceptance of high-confidence extractions. With a small review…
- Reviewers cannot trace most statistics in a research system's final reports back to a…
- A document-analysis subagent finds two credible studies reporting different adoption…
- A synthesis agent reports a "major contradiction" between a survey finding 22% adoption…