NCA-GENM domain - 25% of the exam

Experimentation

Experimentation is 25% of the NCA-GENM exam. These are the objectives it covers, each with practice questions, with every answer explained.

The domain in numbers

67
Practice questions
4
Objectives
49
Medium
18
Hard

Where people go wrong

  • Raising guidance to restore missing information

    The wrong answers raise the guidance scale whenever an image misses something: trailing prompt details, one specific handbag, or a colour bound to the wrong object. Guidance only amplifies conditioning that already reached the denoiser. Truncated trailing tokens never reached the embedding, so nothing about them can be amplified. A specific object needs an image embedding injected as conditioning, and a colour swap needs the cross-attention maps steered toward the intended regions. Higher scales also add over-saturation.

    See it in a sample question
  • Confusing classifier guidance with classifier-free guidance

    The wrong answers describe classifier guidance, which needs an external noise-robust classifier, and present it as the classifier-free method. Classifier-free guidance uses one network trained with the text condition replaced by a null embedding for a fraction of steps, so it produces both conditional and unconditional predictions. The guided output extrapolates from the unconditional prediction toward the conditional one, and the scale sets how far past it the result moves. A conditional pass alone gives no direction to push along, and a plain average adds no extra prompt strength.

    See it in a sample question
  • Swapping recall at k, MRR and nDCG

    The wrong answers assign one retrieval metric the job of another. Recall at k counts how many relevant passages land in the top-k and ignores order. Mean reciprocal rank looks only at the rank of the first relevant result. Normalised discounted cumulative gain applies a position discount to graded relevance across the whole list, so it fits a 0 to 3 scale where the best passages belong at the top. Mean average precision treats relevance as binary and gives no extra credit for a higher grade.

    See it in a sample question
  • Attaching one decoding control to another's effect

    The wrong answers credit a decoding control with a different control's behaviour. Temperature divides the logits before the softmax, so raising it flattens the whole distribution and lowering it sharpens it. Top-k caps the pool at a fixed count of tokens, while top-p keeps the smallest set whose cumulative probability reaches the threshold, so its pool size varies by step. A repetition penalty is the control that scales down logits of tokens already generated, and beam search does not penalise repeats.

    See it in a sample question
  • Letting vector length decide similarity

    The wrong answers let vector length leak into a ranking meant to judge meaning by direction. Min-max scaling distorts the geometry, zero-padding leaves the L2 norm unchanged, and mean-centring does not equalise lengths, so none of them removes the bias on a dot product. Euclidean and Manhattan distance also respond to magnitude. L2-normalising every embedding to unit length makes the inner product equal cosine similarity, so the ranking depends on the angle alone and the original norms no longer count.

    See it in a sample question

Objectives in this domain

Sample questions from this domain

Free sampleExperimentationmedium

A pipeline conditions an image generator on a long, detailed prompt. The text encoder has a fixed maximum token length, and prompts that exceed it are silently truncated before the embedding is produced. Users report that trailing details in their long prompts, such as background and lighting, are consistently ignored in the generated images. Which change most directly addresses the root cause of the ignored details?

  • ASplit the prompt into segments within the token limit, encode each segment, and combine their embeddings so all details are represented in the conditioning Correct
  • BRaise the classifier-free guidance scale so the conditioning embedding exerts a stronger pull on every denoising step
  • CIncrease the number of denoising timesteps so the model has more iterations to render the requested details
  • DLower the sampler's noise level at the final steps so faint background features are preserved rather than blurred
When prompt truncation drops detail before encoding, the fix is to keep all content within the encoder's token limit, not to amplify or refine an incomplete embedding. Conditioning can only steer generation toward information that survives into the embedding; if the encoder truncates trailing tokens, those instructions are absent from the vector, so the cure is to encode the full prompt in segments and combine the embeddings rather than adjust downstream sampling.

Why A is correct: The trailing tokens were dropped before encoding, so the fix is to keep them inside the encoder's limit by segmenting and encoding each part, then merging the embeddings so background and lighting are actually present in the conditioning signal.

Why B is wrong: Stronger guidance does make the model adhere more to its conditioning and tempts those who treat weak adherence as the problem, but the trailing details never reached the embedding, so amplifying it cannot restore information that was already discarded.

Why C is wrong: More steps refine detail that the conditioning already specifies and seems like a fidelity lever, but extra iterations cannot synthesise instructions that were truncated away and never embedded, so the missing details stay missing.

Why D is wrong: Reducing late-step noise can sharpen fine texture and feels relevant to faint background elements, but the issue is absent conditioning information, not blurring, so this addresses a different failure than the silent truncation.

Free sampleExperimentationhard

A team trains a text-conditional diffusion denoiser and wants to use classifier-free guidance at inference so they can trade prompt fidelity against diversity with a single scale knob. Their training loop only ever passes the real text embedding to the U-Net. At sampling time, raising the guidance scale produces no usable steering and the outputs barely respond to the prompt. Which change to the training procedure is required to make classifier-free guidance work as intended?

  • ATrain a separate noise-aware image classifier on the noisy latents and use its gradients to push samples toward the target class at each reverse step.
  • BIncrease the number of forward diffusion timesteps and switch from a linear to a cosine noise schedule so the unconditional signal emerges naturally.
  • CRandomly replace the text condition with a learned null or empty embedding for a fixed fraction of training steps, so the same network learns both conditional and unconditional denoising. Correct
  • DNormalise every text embedding to unit L2 norm before injection so the conditional and unconditional magnitudes match during sampling.
Classifier-free guidance requires training the denoiser to predict both conditionally and unconditionally by randomly dropping the condition during training. Classifier-free guidance computes a guided prediction by extrapolating along the vector from the unconditional to the conditional denoiser output. A single network supplies both only if it was trained with the condition randomly replaced by a null embedding, so the unconditional branch exists. Without condition dropout there is no unconditional estimate and the guidance difference collapses, leaving prompt steering ineffective.

Why A is wrong: This is classifier guidance, the earlier approach that classifier-free guidance was designed to replace; it is tempting because it also steers samples, but it requires an extra noise-robust classifier and does not make the denoiser itself usable in the classifier-free formulation the team has chosen.

Why B is wrong: A cosine schedule and more timesteps can improve overall sample quality, which makes this sound plausible, but neither produces an unconditional prediction; the schedule never teaches the network to denoise with the condition removed, so guidance still has no baseline to extrapolate from.

Why C is correct: Classifier-free guidance forms its steering vector from the difference between the conditional and unconditional predictions of one shared network, so the denoiser must be trained to produce both by dropping the condition on a fraction of steps; without that the unconditional branch is undefined and guidance has nothing to extrapolate from.

Why D is wrong: Embedding normalisation can stabilise conditioning magnitudes and so seems related to the symptom, but it still only ever feeds a real condition; the model never learns an unconditional mode, so the guided difference remains close to zero and the prompt response stays weak.

Free sampleExperimentationmedium

An evaluation harness for a document retrieval system computes a single number that rewards placing the first relevant passage as high as possible in the ranked list, averaged over all queries. The team cares specifically about how quickly users reach one correct answer, not about retrieving every relevant passage. Which evaluation metric best matches this requirement for tuning their embedding model?

  • AMean reciprocal rank, because it averages the reciprocal of the rank position of the first relevant result across queries, rewarding models that surface a correct answer early. Correct
  • BRecall at k, because it measures the fraction of all relevant passages that appear in the top k results, which captures how early the first correct answer is surfaced for the user.
  • CCosine similarity of the query embedding to the gold passage, because a higher pairwise similarity score directly reflects the ranking quality across the whole result list.
  • DNormalised discounted cumulative gain, because it discards graded relevance and reduces to counting only the single first relevant hit per query in the ranked list.
Choose mean reciprocal rank when the objective is surfacing the first correct result early, distinct from recall which measures completeness. Mean reciprocal rank is defined as the average over queries of 1 divided by the rank of the first relevant result. This makes it sensitive only to where the first correct passage appears, which is exactly the early-answer objective, unlike recall at k which counts how many relevant items appear regardless of order.

Why A is correct: Correct: mean reciprocal rank uses 1 divided by the rank of the first relevant item, so it directly measures how high the first correct passage appears, matching the stated early-answer goal.

Why B is wrong: Tempting because recall at k is a standard retrieval metric, but it counts how many relevant items are retrieved, not the rank of the first one; it ignores ordering within the top k and rewards completeness instead.

Why C is wrong: Tempting because cosine similarity drives the ranking, but a single-pair similarity is not a list-level ranking metric and says nothing about where the first relevant passage lands among distractors.

Why D is wrong: Tempting because normalised discounted cumulative gain is rank-aware, but it uses graded relevance across the whole list rather than discarding it, so the description is wrong and it does not reduce to a first-hit measure.

Free sampleExperimentationmedium

Two engineers discuss how temperature affects the next-token distribution of a transformer language model during sampling. Temperature scales the logits by dividing them before the softmax is applied. Which statement most accurately describes the effect of raising the temperature above 1.0 on that distribution?

  • AIt flattens the probability distribution, raising the relative likelihood of lower-scoring tokens and increasing output diversity. Correct
  • BIt sharpens the probability distribution, concentrating mass on the single highest-scoring token and reducing diversity.
  • CIt removes all tokens whose cumulative probability falls outside a chosen mass threshold before renormalising.
  • DIt fixes the candidate pool to a constant number of highest-scoring tokens regardless of their probability spread.
Understand that temperature above 1.0 flattens the softmax over next-token logits, increasing the diversity of generated text. Softmax over logits divided by temperature T produces sharper distributions as T falls below 1 and flatter ones as T rises above 1, because larger T compresses the differences between logits before exponentiation, redistributing probability mass toward lower-scoring tokens.

Why A is correct: Dividing logits by a value greater than one shrinks the gaps between them, so the softmax produces a flatter distribution that lifts low-probability tokens and yields more varied, less predictable text.

Why B is wrong: This is the effect of temperature below 1.0, not above it; candidates who confuse the direction of the scaling pick this, but raising temperature flattens rather than sharpens the distribution.

Why C is wrong: This describes nucleus (top-p) sampling, a separate truncation step; it is tempting because both control diversity, but temperature rescales logits rather than truncating the candidate set.

Why D is wrong: This describes top-k sampling, which caps the candidate count; it sounds related because it also shapes sampling, but temperature alters the shape of the whole distribution, not the size of a fixed pool.

Free sampleExperimentationmedium

An engineer normalises every refined embedding to unit length before storing it in a vector index that ranks results by inner product. Conceptually, what does this unit-length normalisation make the inner-product ranking equivalent to?

  • ARanking by cosine similarity, because the inner product of two unit vectors equals the cosine of the angle between them. Correct
  • BRanking by Euclidean distance squared, because normalised vectors all lie on a sphere where squared distance and inner product are unrelated.
  • CRanking by raw token overlap, because unit normalisation projects each embedding onto its most active dimension.
  • DRanking by vector magnitude, because after normalisation the inner product is dominated by whichever vector had the larger original norm.
Normalising embeddings to unit length makes inner-product ranking equivalent to cosine similarity, since the dot product of unit vectors equals their cosine. Cosine similarity is the dot product divided by the product of norms. Setting every norm to one makes that denominator equal to one, so the inner product alone equals the cosine of the angle. This is why systems that index by inner product normalise embeddings first to obtain angle-based, magnitude-invariant ranking.

Why A is correct: When both vectors have unit norm, their inner product reduces to the cosine of the angle between them, so an inner-product index over normalised embeddings produces the same ordering as cosine similarity.

Why B is wrong: On the unit sphere, squared Euclidean distance is a decreasing function of the inner product, so they are tightly related rather than unrelated; the clause contradicts itself and is wrong, though the sphere framing sounds plausible.

Why C is wrong: Normalisation rescales the whole vector and does not project onto a single dimension or recover token overlap, so this misdescribes both the operation and its effect; it may tempt by invoking lexical matching.

Why D is wrong: Normalisation removes magnitude entirely by setting every norm to one, so the inner product cannot be dominated by original norms; this is the opposite of what normalisation achieves.

Other domains in this exam

See also the NCA-GENM cert hub, the study guide, and the cheat sheet.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.