AIF-C01 - Generative AI fundamentals (24% of the exam) - Section 2.2
Explain how sampling parameters such as temperature and top-p affect model output.
Understand how temperature and top-p trade off determinism against variety, and why temperature 0 produces greedy, reproducible output. Choose sampling settings to match a requirement such as a reproducible audit summary.
TemperatureTop-p samplingGreedy decoding
Practice question for this objective
lock_openFree sampleGenerative AI fundamentalsmedium
A compliance audit needs a hosted language model to return identical text whenever the same prompt and model version are replayed, so that archived answers can be reproduced exactly on demand. The team controls only the sampling settings. Which two configuration changes each make the output deterministic? Select TWO.
ARaise the temperature toward 1.0 so the model weighs a broader spread of candidate tokens at each step.
BSet the temperature to 0 so the model selects the single highest-probability token at every step.check_circle Correct
CSet top-p to a value just above 0 so the nucleus collapses to only the single most-probable token.check_circle Correct
DRaise top-p toward 1.0 so the model can draw from the entire probability distribution each step.
EIncrease the maximum output-token limit so each archived answer has room to finish in full.
Both temperature 0 and a near-zero top-p collapse token selection to the single most-probable token, making a model's output reproducible. Determinism requires removing the random draw from token selection. Temperature 0 forces greedy decoding, and a top-p just above 0 shrinks the nucleus to the top token alone; either leaves exactly one eligible token at each step, so a fixed prompt and model version reproduce identical text.
Why A is wrong: A temperature near 1.0 flattens the distribution and increases run-to-run variation, which is the opposite of the reproducibility the audit requires.
Why B is correct: Temperature 0 forces greedy decoding, picking the most-probable token each step, so a fixed prompt and model version yield the same text every time.
Why C is correct: A top-p that low keeps only the top token in the candidate set, leaving no alternatives to sample, which produces the same deterministic continuation each run.
Why D is wrong: A top-p near 1.0 leaves the full distribution available for sampling, so repeated calls can still diverge rather than reproducing identical text.
Why E is wrong: The output-token limit governs how long a response may run, not whether the same prompt reproduces the same wording, so determinism is unaffected.
Answers that look right on this material and are not. Each one is a distractor from a different question in the AIF-C01 bank for this domain.
Set top-p to 1.0 so the model can draw from every available token at each step.
errorWhy it is wrong: A top-p of 1.0 leaves the full distribution available for sampling, so repeated calls can still diverge and the cache would rarely hit.
Set temperature to 1.0, the neutral midpoint, so the model neither amplifies nor dampens the token probabilities it was trained on.
errorWhy it is wrong: Temperature 1.0 leaves the trained probabilities unscaled but still samples from the full distribution, so successive runs vary; it is tempting because 1.0 sounds like a stable default but it does not force a single deterministic choice.
Both deployments will return identical text on repeat calls, because the model weights are fixed and the decoding strategy does not affect reproducibility.
errorWhy it is wrong: Fixed weights do not guarantee identical text once sampling is involved; this is tempting because the model itself is unchanged, but the decoding strategy determines whether token selection is deterministic.