CCAR-F scenarios › Structured Data Extraction · Question 60 of 60
Scenario 6 · Structured Data Extraction
You run a system that uses Claude to turn unstructured documents into structured records, validates every result against JSON schemas, and passes the output to downstream systems. Accuracy has to stay high across messy and unusual documents, and edge cases must be handled cleanly rather than with plausible-looking guesses.
Context Management & Reliability
During extraction, the agent calls a lookup_vendor tool to match each vendor named in a document to your master data. Each call returns about 60 fields, including addresses, bank history, and audit metadata, but extraction needs only vendor_id, legal_name, and tax_id. On documents with many vendors, extraction accuracy drops noticeably in the second half. What is the most effective change?
Show answer and explanation
Answer: C — Trim each lookup_vendor result to the fields extraction needs, for example in a PostToolUse hook, before it is added to the context.
Verbose tool results build up in context and use tokens out of proportion to their relevance. Section 6 says to trim them to the relevant fields before they accumulate, and a PostToolUse hook is the deterministic place to do it. (a) leaves all 60 fields in context. (b) is an arbitrary cap that pushes work onto people. (d) relies on summaries, which tend to lose exactly the IDs and numbers extraction depends on.
More from this scenario
- Your invoice extraction tool's JSON schema marks purchase_order_number as required…
- A mixed document inbox contains invoices, receipts, and service contracts. You defined…
- Your extraction pipeline validates each result with Pydantic. When validation fails, the…
- Your contract schema has a nullable effective_date. Spot checks show it comes back null…
- A nightly job submits 10,000 scanned-and-OCR'd claim documents through the Message…