NCA-GENL - Trustworthy AI (10% of the exam) - Section 5.2
Understand alignment and ethical considerations in LLM development.
Understand alignment techniques such as reinforcement learning from human feedback (RLHF) and constitutional AI that steer LLM behaviour towards helpful, harmless, and honest outputs. Recognise the ethical risks of LLM deployment - including bias, hallucination, and misuse - and the responsible AI practices that organisations adopt to mitigate them.
AlignmentResponsible AI
Practice question for this objective
lock_openFree sampleTrustworthy AImedium
Many alignment efforts frame their objective using the 'helpful, harmless, and honest' (HHH) criteria. Which statement correctly characterises what these three criteria are intended to specify?
AThree mandatory architectural modules - a helper head, a safety head, and a verifier head - that every aligned transformer must include.
BThree quantitative benchmark scores that must each exceed 0.9 before a model is certified as aligned for release.
CA set of high-level behavioural goals: the assistant should help the user, avoid causing harm, and avoid deceiving the user, which together guide alignment decisions.check_circle Correct
DA data-collection protocol specifying that training corpora must contain equal proportions of helpful, harmless, and honest example sentences.
Identify the helpful, harmless, and honest criteria as high-level behavioural goals that guide LLM alignment rather than architecture or fixed thresholds. The HHH criteria state three behavioural aims for an aligned assistant - being helpful, avoiding harm, and being honest - and serve as a compact normative target that informs training and trade-off decisions, rather than defining model architecture, certification thresholds, or corpus ratios.
Why A is wrong: HHH describes desired behavioural properties, not required architectural components, and no transformer is obliged to contain named heads bearing these labels. It is tempting because the three-part naming suggests three model parts, but the criteria constrain behaviour rather than structure.
Why B is wrong: HHH is a qualitative set of goals, not a trio of fixed numeric thresholds that gate certification. The trap appeals to the desire for crisp pass marks, but no standard assigns a 0.9 cutoff per criterion, and alignment is not certified by three such numbers.
Why C is correct: The HHH framing names three behavioural targets - being helpful to the user, avoiding harmful actions or content, and being honest rather than deceptive - that together act as a compact statement of what an aligned assistant should do. These goals guide trade-off and training decisions, which is exactly their intended role.
Why D is wrong: HHH does not prescribe corpus composition ratios; it describes how the model should behave, not how the dataset must be balanced. It is plausible because dataset curation matters for alignment, but the criteria are behavioural goals rather than a sampling recipe.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENL bank for this domain.
A hallucination problem caused by insufficient grounding data in the fine-tuning corpus.
errorWhy it is wrong: Hallucination refers to a model generating plausible but factually unsupported content and is a failure of the honest dimension of HHH. The scenario describes a conflict between the helpful and harmless criteria when responding to sensitive queries, not a grounding or factuality failure.
RLHF requires only unlabelled text, whereas DPO requires pairs of human-ranked responses to learn the preference signal.
errorWhy it is wrong: This inverts the data requirement: both methods rely on human preference comparisons, and DPO is in fact defined by training directly on ranked pairs. It is tempting because RLHF is associated with large-scale training, but RLHF still needs labelled comparisons to build its reward model.
Helpfulness, because the model is failing to provide the detailed medical guidance the user requested.
errorWhy it is wrong: Helpfulness is tempting because the answers concern usefulness, but the model is actually being too forthcoming rather than unhelpful, so the failure is not a lack of assistance.