CCAR-F scenarios › Claude Code for Continuous Integration · Question 47 of 60
Scenario 5 · Claude Code for Continuous Integration
Claude Code runs inside your CI/CD pipeline. It reviews pull requests automatically, proposes test cases, and posts feedback on the PR. The central challenge is writing prompts and configuration that produce actionable findings while keeping false positives low enough that developers keep trusting the output.
Prompt Engineering & Structured Output
Your review flags issues in four categories: security, correctness bugs, performance, and comment accuracy. About 60% of the comment-accuracy findings are wrong, and developers have started dismissing every automated comment, including valid security findings. What is the best immediate action?
Show answer and explanation
Answer: C — Temporarily disable the comment-accuracy category, then rewrite its criteria to flag only comments whose claimed behaviour contradicts the actual code.
Categories with many false positives erode trust in the accurate ones. Section 6 recommends disabling such a category temporarily while its prompt is improved with explicit criteria, for example flagging only comments that contradict what the code does. (a) still posts the noise and asks developers to sort it out. (b) triples the cost and can suppress real findings that show up only intermittently. (d) is a vague "be careful" instruction that does little for precision.
More from this scenario
- The same kind of problem, an unparameterized SQL string built from request input, is…
- Your pipeline posts a readable markdown summary of review findings on each PR. The…
- Developers dismiss about 30% of automated review findings. The team wants to improve the…
- Your pipeline runs `claude -p` to review each pull request, and a script then uses…
- The CI review job re-runs on every push to an open pull request. Developers complain…