CCAR-P practice › Claude Models, Prompting & Context Engineering · Question 20 of 110

Claude Models, Prompting & Context Engineering

A personal-finance app categorizes every card transaction with Claude, millions of events a month, and the result must appear in the app within a tight latency budget. All traffic runs on the largest model tier. On a labeled set of 2,000 real transactions, both the mid and the smallest tier meet the agreed accuracy bar, with no failure cluster on any merchant type. What is the most appropriate change?

Show answer and explanation

Answer: B — Move the step to the smallest tier that passed, pin its version, and keep sampling production accuracy against the same bar

(b) is right: the step is high volume, latency sensitive and well specified, and the smallest tier passed a task-specific eval with no failure cluster, so it is the right tier; pinning and ongoing sampling guard against drift. (a) pays the largest-tier price and latency on a step that evidence shows doesn't need it. (c) overrides the eval with a guess; if unseen inputs worry you, extend the eval set. (d) adds latency and output tokens to a classification task that already passes without reasoning.