NCA-GENM - Experimentation (25% of the exam) - Section 1.1
Understand how transformer-based LLMs manipulate, analyse, and generate text.
Describe how transformer-based LLMs use attention mechanisms to process and generate text token by token. Recognise how tokenisation, context windows, and autoregressive decoding determine the shape and quality of text output.
TransformersLLMs
Practice question for this objective
lock_openFree sampleExperimentationmedium
An interviewer asks why a transformer language model is described as autoregressive at inference time even though it processes a full prompt in a single forward pass. Which statement best resolves this apparent contradiction?
AThe model generates the entire output sequence in one forward pass and only appears sequential because tokens are displayed to the user one at a time.
BThe prompt is encoded in one pass, but each new output token is generated sequentially and appended to the input that conditions the next token.check_circle Correct
CAutoregressive refers only to training with teacher forcing, and at inference the model is fully parallel with no sequential dependency.
DThe model predicts all tokens simultaneously and then reorders them by probability, which creates the illusion of left-to-right generation.
Understand that transformers encode the prompt in parallel but generate output tokens sequentially, each conditioned on previously produced tokens. Parallel processing applies to known input tokens during the prefill stage, but generation is autoregressive: every new token depends on all tokens generated before it, so the decode stage proceeds one token at a time despite the parallel prompt encoding.
Why A is wrong: Output tokens are genuinely produced one per step, not all at once; the streaming display is real but is a consequence of true sequential generation, not merely a presentation effect.
Why B is correct: Prompt encoding can be parallelised because all prompt tokens are known, yet generation is autoregressive because each produced token is fed back as context for the next, making output generation inherently sequential.
Why C is wrong: Teacher forcing is a training technique, but inference remains autoregressive; this tempts candidates who associate the term solely with training, yet the sequential dependency persists at generation time.
Why D is wrong: There is no reordering step; tokens are produced left to right with each conditioned on prior ones, so the described simultaneous prediction and reordering does not occur in autoregressive decoding.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENM bank for this domain.
Layer normalisation applied before each sub-layer, which scales activations and prevents the model from amplifying future-position signals
errorWhy it is wrong: Layer normalisation stabilises training by normalising activations and has no mechanism for masking positional access; it operates on the magnitude of values, not on which positions are attended to.
Greedy decoding samples each token randomly from the full distribution, whereas beam search always selects the single most probable token at each step.
errorWhy it is wrong: This inverts both definitions: greedy decoding is deterministic, not random, and beam search tracks multiple candidates rather than one; it tempts candidates who remember that the two differ but misattribute which is which.
Increasing the temperature, which flattens the probability distribution and makes the model less likely to select the highest-probability token repeatedly
errorWhy it is wrong: Higher temperature does increase output diversity by distributing probability more evenly, but it operates uniformly on all tokens and does not specifically penalise tokens that have already appeared; it can reduce repetition indirectly but is not the targeted mechanism for this problem.