An evaluation lead is assembling a plan to assess a foundation model that will power a clinical-summary feature. The lead wants the plan to combine evaluation methods so no single number decides the verdict and the assessment mirrors the real task. Which two statements about choosing evaluation methods are correct? Select TWO.
- AA public benchmark gives a comparable, repeatable score but can overstate true ability if its items already appeared in the model's training data. Correct
- BA task-specific metric computed on a held-out set built from real clinical summaries tracks the deployed task more closely than a generic benchmark does. Correct
- CA single automated accuracy figure on any held-out set is enough on its own to decide the verdict once it clears a chosen pass threshold.
- DHuman expert review should be dropped from the plan here, because clinician judgements are slower and more costly than running automated scoring.
- EReporting only the overall mean score across all cases is preferable to also inspecting performance on the rare but high-stakes summary types.
Why A is correct: Standardised benchmarks aid comparison yet are open to contamination, so a high score can reflect memorisation rather than genuine capability.
Why B is correct: Evaluating on data drawn from the actual task narrows the gap between test and production, making the result a more faithful proxy for live behaviour.
Why C is wrong: One aggregate number hides failure modes and task mismatch, which is why the plan should combine complementary methods rather than trust a threshold.
Why D is wrong: Cost and speed do not make expert review unsound; for clinical correctness it captures quality that automated overlap scores miss, so it stays in.
Why E is wrong: An overall mean can mask poor performance on rare critical cases, so slicing the results is needed rather than collapsing everything to one figure.