CCAR-P practice › Governance, Safety & Risk Management · Question 76 of 110
Governance, Safety & Risk Management
An insurer's policy Q&A assistant has run well for months. The week after the content team migrated the knowledge base to a new document management system, customer complaints rise about answers that cite deductible amounts confidently but wrongly. Model version, prompts and latency are unchanged. What is the most appropriate first step?
Show answer and explanation
Answer: D — Inspect the retrieved chunks for failing queries for parsing, chunking or index-freshness breaks from the migration
The symptom lines up with a change in the retrieval layer, and the model and prompt didn't change. Inspecting the context the model actually received shows whether tables or numbers were mangled in migration. (a), (b) and (c) treat it as a model problem. None of them would fix wrong or stale content being fed into the context, and (c) adds cost without addressing the cause.
Keep going
More Governance, Safety & Risk Management questions
- You are building a risk register for an agent that handles accounts-payable exceptions…
- A multi-step research agent compiles supplier risk reports. Reviewers notice that when…
- A healthcare payer's prior-authorization assistant proposes approve or deny…
- A logistics company's agent can reschedule deliveries, issue refunds up to a configured…
- A team implemented an approval step for high-value contract amendments: the agent drafts…