An interviewer asks why a transformer language model is described as autoregressive at inference time even though it processes a full prompt in a single forward pass. Which statement best resolves this apparent contradiction?
- AThe model generates the entire output sequence in one forward pass and only appears sequential because tokens are displayed to the user one at a time.
- BThe prompt is encoded in one pass, but each new output token is generated sequentially and appended to the input that conditions the next token. Correct
- CAutoregressive refers only to training with teacher forcing, and at inference the model is fully parallel with no sequential dependency.
- DThe model predicts all tokens simultaneously and then reorders them by probability, which creates the illusion of left-to-right generation.
Why A is wrong: Output tokens are genuinely produced one per step, not all at once; the streaming display is real but is a consequence of true sequential generation, not merely a presentation effect.
Why B is correct: Prompt encoding can be parallelised because all prompt tokens are known, yet generation is autoregressive because each produced token is fed back as context for the next, making output generation inherently sequential.
Why C is wrong: Teacher forcing is a training technique, but inference remains autoregressive; this tempts candidates who associate the term solely with training, yet the sequential dependency persists at generation time.
Why D is wrong: There is no reordering step; tokens are produced left to right with each conditioned on prior ones, so the described simultaneous prediction and reordering does not occur in autoregressive decoding.