An HR officer at a supermarket chain asks Claude to summarise 300 free-text staff survey responses about a proposed new shift pattern. The summary says staff 'overwhelmingly oppose' the change and supports this with four strongly worded quotes. The board will decide whether to go ahead based on the summary. Which check would best show whether the summary gives a fair picture of staff views?
- AConfirm that each of the four quotes appears word for word in the survey responses before the summary goes to the board.
- BAsk Claude whether its summary is balanced, and to rewrite any parts it judges one-sided before it goes to the board.
- CAsk Claude to soften the wording to neutral phrasing, such as 'many staff have concerns', before sending it.
- DCount how many responses oppose, support or are neutral, and compare that split with the summary's 'overwhelmingly oppose'. Correct
Why A is wrong: Checking quotes against the source is good practice and catches fabricated wording, so this feels thorough. But genuine quotes can still be the most extreme few out of 300, so this check cannot show whether 'overwhelmingly oppose' is true.
Why B is wrong: Asking for a self-review is quick and sounds responsible. It is not independent evidence, though: Claude is judging its own selection without anyone counting the responses the summary is supposed to represent.
Why C is wrong: Toning down strong language seems like a fix for bias. But rewording without checking the data only swaps one unverified claim for another, and 'many staff have concerns' could still misrepresent a workforce that mostly supports the change.
Why D is correct: The claim at risk is about proportion, so the check must measure proportion. Tallying the responses tests whether the summary's strongest word and its chosen quotes reflect the whole survey or a vocal minority.