Claude Certification Guide CCAR-F

Curriculum › Prompt Engineering & Structured Output · 20% of the exam

Multi-instance and multi-pass review

What you’ll be able to do

  • Explain why same-session self-review misses issues an independent instance catches
  • Set up a second Claude instance to review generated code without the generator’s reasoning context
  • Split a large multi-file review into per-file local passes plus a cross-file integration pass
  • Add a verification pass that reports confidence per finding to drive review routing

What you need to know

Why self-review underperforms

When a model generates code and is then asked, in the same session, to review it, the review starts from the same reasoning that produced the code. The assumptions that led to a bug are still sitting in context, framed as decisions already made. The model is less likely to question those decisions than a reader seeing the code fresh, so it tends to confirm its own work and miss exactly the subtle issues a review exists to catch.

Instructions don't remove this. "Critically review your previous answer" or "pretend you're a skeptical senior engineer" still runs over a context full of the generator's reasoning. Extended thinking in the same session has the same limit: more reasoning over the same framing isn't a fresh perspective.

Independent instances catch what self-review misses

An independent review instance is a separate Claude call that sees the artifact (the code, the diff, the extraction) and the review criteria, but none of the generator's reasoning. Without that context it reads the code as it is, not as it was intended, which is why independent review is more effective at catching subtle issues than self-review instructions or extended thinking.

before — self-review in the generating session
messages.append({"role": "user", "content": "Now carefully review the code you just wrote."}) review = client.messages.create(messages=messages, ...) # full generation history
after — independent reviewer with no generation history
review = client.messages.create( system=REVIEW_CRITERIA, # explicit report/skip criteria (4.1) messages=[{"role": "user", "content": f"Review this diff:\n{diff}"}], # artifact only ...)

The reviewer gets the artifact and the criteria. Don't hand it the generator's explanation of why the code is right "for context"; that brings back the bias you're trying to remove.

Large reviews: split into per-file and integration passes

A single pass over a 15-file pull request asks the model to hold every file in view at once. The predictable result is attention dilution: thorough comments on some files, superficial ones on others, obvious bugs missed. It also produces contradictory findings, such as flagging a pattern in one file while approving identical code in another. A bigger context window doesn't fix this, because the problem is how attention is spread, not whether the files fit.

Restructure as multiple passes:

  • Per-file local passes. Review each file on its own for local issues: logic errors, unhandled cases, security problems within that file. Each pass gets full attention on one file, so depth is consistent.
  • Integration pass. A separate pass focused on cross-file concerns: data flow between modules, a changed function signature and its callers, an assumption one file makes about another. Give it the interfaces and the relevant per-file summaries rather than every line again.

Per-file passes alone miss cross-file bugs; a single combined pass dilutes attention. You need both. This is also better than the tempting alternatives: asking developers to split their PRs shifts the burden without improving the reviewer, and running the same full-PR review several times and keeping only findings that recur suppresses real bugs that are only caught some of the time.

Verification passes with per-finding confidence

After findings are produced, a verification pass can re-examine each one and have the model self-report a confidence alongside it. The point isn't to discard low-confidence findings automatically. It's to route them: high-confidence findings post directly, lower-confidence ones go to a human reviewer or a further check. Self-reported confidence is only useful for routing once you've checked it against outcomes, such as which findings were accepted or dismissed, so set the thresholds from that data rather than picking a number (5.5 covers calibration). And confidence routing complements explicit review criteria (4.1); it doesn't replace them.

finding shape from the verification pass
{"location": "src/orders/sync.py:142", "issue": "retry loop re-sends a non-idempotent POST", "severity": "MAJOR", "confidence": 0.62, # below the calibrated threshold → human review "detected_pattern": "retry around non-idempotent call"}

Key concept

A model reviewing its own output in the same session is anchored by its own reasoning, so use an independent instance that sees only the artifact and the criteria. Split large reviews into per-file local passes plus an integration pass for cross-file data flow, and use per-finding confidence from a verification pass to route findings, with thresholds calibrated against real outcomes.

When a scenario describes inconsistent depth or contradictory findings across a large change, the fix is multi-pass structure. When it describes generated code whose bugs slip past "review your work," the fix is an independent reviewer.

Practice scenario

ScenarioA CI pipeline uses one Claude session to generate unit tests for a change and then, in the same session, asks it to "double-check the tests for mistakes" before committing them. Tests that assert the buggy behavior of the code under test keep getting committed. The team's proposed fix is to enable extended thinking for the double-check step.
Work it through, then open this

The double-check runs with the generator’s full reasoning in context, including its belief about what the code is supposed to do. That’s why it doesn’t question assertions that match the bug. Extended thinking adds more reasoning over the same framing, not an independent view. Send the generated tests and the code under test to a separate Claude instance, with no generation history, and ask it to check each assertion against the intended behavior described in the spec or ticket. The independent reviewer reads the tests fresh and is far more likely to notice an assertion that encodes a bug.

Build exercise — Restructure a review into independent, multi-pass stages

Intermediate · 40 min

What you’ll learn

  • Separating generation and review into independent instances
  • Splitting a large review into per-file and integration passes
  • Adding a confidence-reporting verification pass for routing

  1. Take a generate-then-self-review flow and move the review into a separate call that receives only the artifact and explicit review criteria.

    • Why: Removing the generator’s reasoning removes the anchor that makes self-review confirm its own decisions.
    • You should see: The independent reviewer flagging issues the self-review passed, on the same inputs.

  2. For a pull request of 10+ files, replace the single review pass with one pass per file for local issues plus one integration pass over interfaces and cross-file data flow.

    • Why: Per-file passes give consistent depth; the integration pass catches what no single-file view can.
    • You should see: Consistent depth across files, no contradictory verdicts on identical code, and at least one cross-file issue surfaced only by the integration pass.

  3. Add a verification pass that attaches a confidence to each finding, and route findings by threshold. Set the threshold by comparing confidence with which findings developers accepted.

    • Why: Confidence is useful for routing only once it’s checked against real outcomes.
    • You should see: High-confidence findings mostly accepted, and the lower-confidence band concentrating the dismissals, which is the band worth sending to humans.

Exam traps

Asking the generating session to “review your work carefully” as the quality gate

The session retains its own reasoning and is less likely to question its decisions. Use an independent instance.

Relying on extended thinking in the same session instead of an independent reviewer

More reasoning over the same context isn’t a fresh perspective. Independent instances catch subtle issues more effectively.

Passing the generator’s reasoning to the reviewer “for context”

That reintroduces the bias the independent instance is meant to remove. Give the reviewer the artifact and the criteria.

Reviewing many files in one pass and expecting consistent depth

Attention dilutes across files, producing uneven depth, missed bugs, and contradictory findings. Split into per-file passes.

Moving to a larger context window to fix attention dilution

Fitting the files isn’t the problem; attention quality is. Multi-pass structure addresses it, a larger window doesn’t.

Keeping only findings that several identical full-PR runs agree on

Consensus filtering suppresses real bugs that are only caught some of the time. Restructure the passes instead.

Running per-file passes only, with no integration pass for cross-file issues

Data-flow and interface bugs span files. A separate integration pass is needed to catch them.

Sources

Quick check

Select TWO.

A test-generation reviewer is inconsistent about whether an untested error branch counts as a coverage gap worth reporting. The team wants consistent judgment that also carries over to cases it hasn't seen. Which additions to the prompt best achieve this? (Select two)