A recruiting platform uses Claude to rank applicants against job descriptions. To address bias concerns, the team removed name, age and gender from the input. Overall ranking quality looks strong on the eval set. What is the best next step to assess fairness?

Answer: B — Test counterfactual pairs varying proxies and report outcomes per segment against an agreed criterion

Removing explicit attributes doesn't remove proxies, and aggregate quality hides group-level differences. Counterfactual tests and disaggregated metrics against a deliberately chosen criterion measure the effect directly. (a) is guidance, not a control or evidence. (c) sharpens the wrong metric. (d) may strip job-relevant information and still hasn't measured anything.