CCAR-F scenarios › Structured Data Extraction · Question 51 of 60
Scenario 6 · Structured Data Extraction
You run a system that uses Claude to turn unstructured documents into structured records, validates every result against JSON schemas, and passes the output to downstream systems. Accuracy has to stay high across messy and unusual documents, and edge cases must be handled cleanly rather than with plausible-looking guesses.
Prompt Engineering & Structured Output
Your invoice extraction tool's JSON schema marks purchase_order_number as required. Downstream reconciliation finds that on invoices with no PO, the output contains values like "PO-0000" or a copy of the invoice number. Schema validation passes on all of them. What change best fixes this?
Show answer and explanation
Answer: C — Make purchase_order_number optional and nullable in the schema, and instruct Claude to return null when the document contains no purchase order number.
A required field pushes the model to fabricate a value when the source doesn't contain one. Making the field optional or nullable removes that pressure, as section 6 describes. (a) keeps the pressure and adds a sentinel string downstream systems have to special-case. (b) retries for information that isn't in the document, which can't succeed, and invented values can still match a format check. (d) moves the field into a separate call but leaves the same required-field pressure in place.
More from this scenario
- A mixed document inbox contains invoices, receipts, and service contracts. You defined…
- Your extraction pipeline validates each result with Pydantic. When validation fails, the…
- Your contract schema has a nullable effective_date. Spot checks show it comes back null…
- A nightly job submits 10,000 scanned-and-OCR'd claim documents through the Message…
- Select two. Your extraction system scores 97% field-level accuracy on last quarter's…