CCAR-F scenarios › Structured Data Extraction · Question 60 of 60

Scenario 6 · Structured Data Extraction

You run a system that uses Claude to turn unstructured documents into structured records, validates every result against JSON schemas, and passes the output to downstream systems. Accuracy has to stay high across messy and unusual documents, and edge cases must be handled cleanly rather than with plausible-looking guesses.

Context Management & Reliability

During extraction, the agent calls a lookup_vendor tool to match each vendor named in a document to your master data. Each call returns about 60 fields, including addresses, bank history, and audit metadata, but extraction needs only vendor_id, legal_name, and tax_id. On documents with many vendors, extraction accuracy drops noticeably in the second half. What is the most effective change?

Show answer and explanation

Answer: C — Trim each lookup_vendor result to the fields extraction needs, for example in a PostToolUse hook, before it is added to the context.

Verbose tool results build up in context and use tokens out of proportion to their relevance. Section 6 says to trim them to the relevant fields before they accumulate, and a PostToolUse hook is the deterministic place to do it. (a) leaves all 60 fields in context. (b) is an arbitrary cap that pushes work onto people. (d) relies on summaries, which tend to lose exactly the IDs and numbers extraction depends on.