A logistics company routes all customer email through a large, capable model that classifies each message into one of six categories. A cost review asks whether this is the right model. The current accuracy is 97% on the eval set. What is the best way to determine whether there is a model mismatch?

Answer: A — Run the same eval and prompt on a smaller model and compare per-category accuracy and cost per correct result

(a) is right: model mismatch can mean over-provisioning, and the way to establish it is evidence: the same eval and prompt on another tier, compared per category and per correct classification. (b) assumes high accuracy proves the model choice is efficient. (c) downsizes without evidence, risking specific categories. (d) relies on self-reported confidence, which isn't a calibrated measure of whether a smaller model would do as well.