A healthcare provider's patient-intake assistant passed launch review with a zero rate of disallowed medical advice on the safety test set. Two weeks after launch, nurses report that patients are abandoning the intake flow because the assistant declines to answer routine questions such as clinic hours and how to prepare for a blood test. What gap in the evaluation metrics most likely allowed this?

Answer: D — No metric tracked over-refusal of legitimate requests, so helpfulness was traded away unseen

(d) is right: safety evaluation has two directions. Driving harmful output to zero is often achieved by refusing too much, and without an over-refusal metric on legitimate questions that trade-off is invisible until users complain. (a) concerns the harmful-output metric, which isn't what failed. (b) doesn't fit a symptom of explicit refusals. (c) could affect harmful-output scoring but wouldn't create a helpfulness collapse on routine questions.