During a briefing on how generative models produce output, an engineer wants to explain what happens at inference time when a deployed language model answers a prompt. Which statement best describes how the model generates its reply?
- AIt retrains its own internal weights on the user's prompt before it returns any generated text.
- BIt searches a fixed database of stored answers and returns the closest matching record it finds.
- CIt predicts the next token repeatedly, each time using the prompt and the tokens it has already produced. Correct
- DIt converts the prompt into one numeric vector and then returns that vector as its final reply.
Why A is wrong: This is tempting because prompts do influence the output, but inference does not change the model's weights; training and inference are separate, so no retraining happens per prompt.
Why B is wrong: This sounds like retrieval, which makes it plausible, but a generative model composes new text by predicting tokens rather than looking up a stored answer, so the description is wrong.
Why C is correct: Correct: at inference the model generates one token at a time, feeding the prompt plus its own earlier tokens back in to predict each following token until the reply is complete.
Why D is wrong: Producing a vector is what an embedding model does; it is tempting because vectors are involved internally, but a language model returns generated text, not a raw vector.