NCA-GENM - Multimodal Data (15% of the exam) - Section 3.1
Generate images from English text prompts using CLIP.
Explain how CLIP aligns text and image representations in a shared embedding space, enabling text prompts to guide image generation. Apply prompt engineering techniques to steer CLIP-conditioned outputs toward the desired visual concept.
CLIP
Practice question for this objective
lock_openFree sampleMultimodal Datamedium
A generative image pipeline uses CLIP to evaluate whether candidate images produced by a diffusion model match a given text prompt. Which property of CLIP makes it suitable for this guidance role?
ACLIP is a pixel-level decoder that reconstructs images token by token from a text sequence.
BCLIP is a variational autoencoder that compresses images into a latent distribution conditioned on text.
CCLIP uses autoregressive cross-attention to generate image patches in the order determined by the text prompt.
DCLIP projects both images and text into a shared embedding space where cosine similarity measures semantic alignment.check_circle Correct
Explain how CLIP's shared embedding space enables it to guide text-to-image generation without acting as a generator itself. CLIP (Contrastive Language-Image Pretraining) is trained to maximise cosine similarity between matched image-text pairs and minimise it for unmatched pairs. This contrastive objective produces a joint embedding space where the semantic content of a caption and its corresponding image land close together. A diffusion model or other generator can exploit this by computing the CLIP similarity between its current output and the target text prompt and using the gradient of that score to steer the denoising process toward higher semantic fidelity.
Why A is wrong: CLIP contains no decoder and produces no pixels; it is a dual-encoder trained to score text-image similarity, not to generate or reconstruct images.
Why B is wrong: A variational autoencoder uses encoder-decoder structure with a probabilistic latent space; CLIP uses contrastive learning across two modality-specific encoders with no variational bottleneck.
Why C is wrong: Autoregressive patch generation describes models such as image GPT; CLIP generates no patches and does not process modalities sequentially in a generative manner.
Why D is correct: Contrastive training aligns image and text embeddings so that a matched pair scores high similarity, enabling CLIP to act as a differentiable semantic signal for guiding a separate generator.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENM bank for this domain.
Pixel-level reconstruction loss between the original image and a decoded version of the text embedding.
errorWhy it is wrong: Pixel reconstruction is the objective of autoencoders and VAEs; CLIP never decodes embeddings back to pixels and uses no reconstruction loss in its pre-training.
Because CLIP outputs a calibrated probability for each image that sums to one across the candidate set, the highest probability is the best match.
errorWhy it is wrong: Tempting because softmax over candidates is a common pattern, but wrong: raw CLIP similarities are not calibrated probabilities, and matching relies on the shared-space similarity itself rather than a normalised distribution.
CLIP is a generative decoder that maps a text embedding directly back into pixel space, so it should output an image on its own once the prompt is encoded.
errorWhy it is wrong: This is tempting because CLIP is central to many text-to-image pipelines, but it is wrong: CLIP has no pixel decoder and learns only to align image and text embeddings, so it cannot synthesise images by itself.