CCAR-P practice › Stakeholder Communication & Lifecycle Management · Question 89 of 110

Stakeholder Communication & Lifecycle Management

A feasibility probe for invoice extraction scored very well on 30 invoices the finance team picked as typical. In the pilot, accuracy drops sharply: many real invoices are scanned, some are in languages nobody mentioned, and a few vendors use multi-page layouts. What was the discovery mistake?

Show answer and explanation

Answer: A — The probe sample wasn't representative of real inputs, so it validated an assumption production didn't share

(a) is right: hand-picked 'typical' examples hide exactly the messy cases production brings; a representative real sample would have exposed scans, languages and layouts during discovery. (b) and (c) are design changes that might matter later, but the process failure was testing on the wrong data. (d) is backwards: a high score on unrepresentative data is the problem, not a high threshold.