CCAR-F scenarios › Structured Data Extraction · Question 51 of 60

Scenario 6 · Structured Data Extraction

You run a system that uses Claude to turn unstructured documents into structured records, validates every result against JSON schemas, and passes the output to downstream systems. Accuracy has to stay high across messy and unusual documents, and edge cases must be handled cleanly rather than with plausible-looking guesses.

Prompt Engineering & Structured Output

Your invoice extraction tool's JSON schema marks purchase_order_number as required. Downstream reconciliation finds that on invoices with no PO, the output contains values like "PO-0000" or a copy of the invoice number. Schema validation passes on all of them. What change best fixes this?

Show answer and explanation

Answer: C — Make purchase_order_number optional and nullable in the schema, and instruct Claude to return null when the document contains no purchase order number.

A required field pushes the model to fabricate a value when the source doesn't contain one. Making the field optional or nullable removes that pressure, as section 6 describes. (a) keeps the pressure and adds a sentinel string downstream systems have to special-case. (b) retries for information that isn't in the document, which can't succeed, and invented values can still match a format check. (d) moves the field into a separate call but leaves the same required-field pressure in place.