Claude Certification Guide CCAR-P

Curriculum › Governance, Safety & Risk Management · 14% of the exam

Human-in-the-loop validation

What you’ll be able to do

  • Choose among pre-action approval, exception routing, sampled post-hoc review and on-the-loop monitoring based on impact and reversibility
  • Design review gates the system enforces, with the evidence reviewers need to make a real decision
  • Counter automation bias and review fatigue so human oversight stays meaningful at scale
  • Use reviewer decisions as a feedback loop that measures quality and justifies widening or narrowing automation

What you need to know

Four placements for a human

  • Pre-action approval (human in the loop). The system proposes and a human approves before anything happens. Use it for irreversible, high-impact or regulated actions: payments above a threshold, account closures, external communications on legal matters, production changes.
  • Exception routing. The system acts autonomously on clear cases and routes the rest to people: low calibrated confidence, contradictory sources, policy gaps, a customer asking for a human, or a case outside the system's validated scope.
  • Sampled post-hoc review. Humans review a stratified sample of completed automated work to measure error rates and spot new error patterns. Use it where individual errors can be corrected after the fact.
  • Human on the loop. People monitor aggregate behavior (dashboards, alerts, drift metrics) and can pause or roll back the system, without approving individual actions.

Most production systems combine all four, tiered by risk. The tiering question is the same one from 5.2: how bad is a single error, can it be undone, and how many people does it touch? A draft reply a human sends anyway needs no gate. A wire transfer does.

Gates must be enforced, not requested

"Ask the user for confirmation before issuing a refund over the limit" in a system prompt is a request, and a confused or manipulated model can skip it. A real gate is structural. The tool that executes the action requires an approval record that only the review interface can create, or the agent's tool only creates a pending request and a separate service executes it after approval. Bind the approval to the exact action: same amount, same recipient, same parameters, so an approved request can't be modified afterward. Record who approved what, when, and on what evidence. That audit trail is often a compliance requirement in its own right (see 5.4).

Make the review a real decision

A reviewer who sees "Recommend: approve" and an Approve button will mostly approve. That's automation bias, and it gets worse with volume and fatigue. Design against it:

  • Show evidence, not just conclusions: the source passages, the extracted fields next to the document region they came from, the tool results the agent relied on, and what's uncertain or conflicting.
  • Highlight what needs judgment. Point the reviewer at the low-confidence field or the contradiction instead of asking them to re-check everything.
  • Keep volume humane. If reviewers face hundreds of near-identical approvals a day, the gate is theater. Narrow the gate to the cases that matter, or automate the clear ones with sampling.
  • Test the reviewers. Seed known-bad items into queues and measure the catch rate. Track override rates: near-zero overrides on a meaningful volume usually means rubber-stamping, not perfection.
  • Separate duties where stakes call for it. The person who requested an action shouldn't be the one who approves it.

Calibrate the routing, then keep sampling

Exception routing depends on a signal that actually correlates with error. Calibrate confidence thresholds against labeled data per segment (document type, field, customer tier), not globally, and don't use raw self-reported confidence. When a segment shows sustained accuracy in sampling, you can widen automation for that segment. When sampling or an incident shows new error patterns, narrow it. Never let sampling drop to zero for automated categories: that sample is how you find out the input distribution has shifted.

Close the feedback loop

Every reviewer correction is a labeled example. Capture it in a structured form (what was wrong, which field, which category of error) and route it into regression evals, few-shot examples and prompt or retrieval fixes. Report the trend to stakeholders: error rate by segment, review volume, time to decision. HITL is a cost, and the feedback loop is how you show it's buying down risk. It's also what lets you shrink the cost responsibly over time.

Capacity is a design constraint

Size review queues against real reviewer hours, define SLAs for decisions, and decide in advance what happens when the queue backs up. Does the system hold the action (safe default for irreversible ones) or proceed under a stricter rule? A gate with no capacity plan turns into either a bottleneck or an informal bypass.

Key concept

Place human review by impact and reversibility, enforce gates in the system rather than in the prompt, give reviewers the evidence to make a real decision, and measure whether they’re catching errors.

Questions often offer "add human approval to everything" as the safe-sounding answer. Prefer the tiered design: gates on the irreversible or high-impact actions, exception routing on calibrated signals, and stratified sampling elsewhere. When a scenario shows high approval rates with errors slipping through, suspect automation bias and missing evidence in the review UI before suspecting the model.

Practice scenario

ScenarioA bank's agent drafts responses to commercial-loan covenant questions and also prepares covenant-waiver letters. Compliance mandated that every output be approved by a credit officer. Six months in, officers approve 99.6% of items with a median review time of about 20 seconds, and an audit finds two waiver letters sent with incorrect covenant thresholds. Leadership asks whether to add a second approver to every item.
Work it through, then open this

Doubling a rubber-stamp process doubles the cost without restoring scrutiny. The 20-second median and the near-total approval rate point to review fatigue and automation bias. Tier the work instead. Waiver letters are high-impact, so keep a mandatory gate, but have the review UI show each threshold next to the clause in the loan agreement it came from, flag any value that differs from the agreement, and require the officer to confirm each threshold rather than click once. Routine covenant Q&A can move to exception routing plus stratified sampling. Seed known errors to measure catch rates, feed corrections into regression evals, and make sure the send step can’t run without an approval record tied to that exact letter version.

Build exercise — Design a tiered review policy

Advanced · 40 min

What you’ll learn

  • Mapping actions to review placements by impact and reversibility
  • Designing an enforced approval gate bound to a specific action
  • Defining metrics that show whether review is working

  1. List every action or output type your system produces and assign each a placement: pre-action approval, exception routing, sampled review or on-the-loop monitoring. Write one line of justification each.

    • Why: Uniform review wastes capacity on low-risk items and starves the items that matter.
    • You should see: A small set of gated actions, most volume under routing or sampling, and every gate justified by irreversibility or impact.

  2. For one gated action, sketch the enforcement: the agent creates a pending request, the review UI shows the evidence, approval creates a record bound to the exact parameters, and only then does an executor service act (illustrative design, not a specific product feature).

    • Why: A gate the model could skip isn’t a gate. Binding approval to parameters prevents approve-then-modify.
    • You should see: No path from the agent to the side effect that bypasses the approval record.

  3. Define four review-health metrics: catch rate on seeded errors, override rate, median review time, and error rate found by post-hoc sampling, each with a threshold that triggers action.

    • Why: Without these, you can’t tell real oversight from rubber-stamping, or justify changing automation levels.
    • You should see: A dashboard spec that would have flagged the scenario’s 99.6% / 20-second pattern.

Exam traps

Requiring human approval on every action, so reviewers start rubber-stamping

Blanket gates create fatigue and automation bias. Gate the irreversible or high-impact actions and use routing and sampling elsewhere.

Implementing the approval step as a prompt instruction the model is asked to respect

A prompt request can be skipped. Enforce the gate in the system so the action can’t execute without an approval record.

Showing reviewers only the model’s conclusion, without the source evidence behind it

Without evidence, reviewers can only trust or distrust the model. Show sources and highlight uncertainty so the review is a real decision.

Stopping review entirely for categories that were automated because they looked accurate

Inputs drift. Keep stratified sampling on automated segments so new error patterns are caught.

Measuring review by throughput rather than by whether reviewers catch planted or known errors

Speed and approval rates say nothing about effectiveness. Seeded-error catch rates and override rates do.

Letting reviewer corrections disappear instead of feeding them back into evaluations and prompts

Corrections are labeled data. Capture them in a structured form and turn them into regression tests and fixes.

Sources

Quick check

Select TWO.

A European retailer's support assistant stores conversation transcripts, writes full prompts to application logs, and indexes resolved tickets into a vector store used for retrieval. A customer submits a GDPR erasure request. Which two steps are needed to handle such requests properly? Select two.