CCAR-F scenarios › Structured Data Extraction · Question 59 of 60

Scenario 6 · Structured Data Extraction

You run a system that uses Claude to turn unstructured documents into structured records, validates every result against JSON schemas, and passes the output to downstream systems. Accuracy has to stay high across messy and unusual documents, and edge cases must be handled cleanly rather than with plausible-looking guesses.

Context Management & Reliability

Your system extracts obligations from each section of a 60-section supplier agreement, then concatenates all the per-section outputs into one input for a final consolidation pass that produces the downstream JSON. Obligations from the first and last sections always appear in the final output, but ones from sections 20–45 are often missing. What change best addresses this?

Show answer and explanation

Answer: A — Put a summary of key findings at the start of the consolidated input, and organize the detailed per-section results under explicit section headers.

This is the "lost in the middle" effect: models handle the beginning and end of long inputs reliably and may drop findings in the middle. Section 6's remedy is to put key findings summaries at the beginning and organize details under explicit section headers. (b) adds room in the window but doesn't change how attention is spread across the input. (c) doubles the cost to work around position effects instead of fixing the input's structure. (d) assumes the output is being cut off, but the middle items are being dropped, not truncated.