Multimodal Data

A two-stage neural text-to-speech component in a conversational pipeline separates an acoustic model from a vocoder. Which description correctly contrasts the responsibilities of these two stages?

• AThe acoustic model converts the spectrogram into the final waveform, while the vocoder predicts the spectrogram from the input text.
• BThe acoustic model transcribes the user's speech to text, while the vocoder generates the language model's textual reply.
• CThe acoustic model predicts an intermediate acoustic representation such as a mel spectrogram from text, while the vocoder converts that representation into the audible waveform.check_circle Correct
• DThe acoustic model aligns audio frames to phoneme labels for training, while the vocoder selects the speaker identity at inference time.

Why A is wrong: This reverses the two roles, which is a common confusion, but the acoustic model works from text to an intermediate representation and the vocoder produces the waveform, so the assignment is backwards.

Why B is wrong: Both halves belong to other stages: transcription is the recognition role and reply generation is the language-model role, so neither describes a component of the synthesis stage.

Why C is correct: Correct: in a two-stage synthesiser the acoustic model maps text or phonemes to a mel spectrogram, and the vocoder transforms that spectrogram into the time-domain waveform that is played back to the user.

Why D is wrong: Forced alignment and speaker selection are auxiliary concerns that sound plausible, but they do not capture the core synthesis flow from text to spectrogram to waveform that defines these two stages.

A team building a text-to-image system describes CLIP as the component that turns an English prompt into pixels. A reviewer corrects them. Which statement most accurately describes CLIP's actual role in a CLIP-guided image generation pipeline?

• ACLIP is a generative decoder that synthesises image pixels directly from a text prompt, so the diffusion or GAN module is only needed for upscaling the output.
• BCLIP is a noise scheduler that controls how Gaussian noise is added and removed across diffusion timesteps during the reverse denoising process.
• CCLIP is a tokeniser that converts prompt characters into integer token ids and otherwise leaves all semantic encoding to the downstream generator network.
• DCLIP is a pair of encoders that map images and text into a shared embedding space, supplying the alignment signal or conditioning embedding that a separate generator uses to produce pixels.check_circle Correct

Why A is wrong: Tempting because CLIP is central to text-to-image work, but it is wrong: CLIP has no pixel decoder and cannot synthesise images, and the generator does far more than upscaling.

Why B is wrong: Plausible to anyone conflating the parts of a diffusion stack, but wrong: noise scheduling is a property of the diffusion sampler, and CLIP plays no role in adding or removing noise.

Why C is wrong: Tempting since CLIP does include a tokeniser step, but wrong: tokenisation is only a preprocessing stage feeding CLIP's text transformer, which produces rich semantic embeddings rather than passing raw ids onward.

Why D is correct: Correct: CLIP comprises an image encoder and a text encoder trained to share an embedding space, and a separate model such as a diffusion U-Net or GAN does the actual pixel synthesis using CLIP's embedding or similarity score.

An organisation has three fully trained specialist models: a fine-tuned vision transformer for product image classification, a fine-tuned BERT variant for product description classification, and a fine-tuned audio classifier for spoken product reviews. Retraining any of the three models is cost-prohibitive. The organisation wants to produce a single combined product category prediction. Which fusion strategy is most appropriate given these constraints?

• AEarly fusion: extract the raw embeddings from each model's first layer, concatenate them, and fine-tune a new shared encoder from scratch on the combined embedding.
• BIntermediate fusion: unfreeze the penultimate layers of each specialist model, concatenate those intermediate feature vectors, and jointly fine-tune a small fusion head on top.
• CLate fusion: collect the predicted class probabilities from each specialist model, then combine them using a lightweight aggregation method such as a weighted vote or stacked classifier trained on held-out validation data.check_circle Correct
• DKnowledge distillation fusion: use all three specialist models as teachers and distil their combined knowledge into a single small student model trained end-to-end on all three modalities.

Why A is wrong: Extracting first-layer activations from three frozen specialist models and feeding them into a new shared encoder would require that new encoder to be trained, which contradicts the cost-prohibitive retraining constraint and does not exploit the models' final predictions.

Why B is wrong: Unfreezing and jointly fine-tuning penultimate layers constitutes partial retraining of the specialist models, which the organisation has explicitly ruled out due to cost. Intermediate fusion here would violate the given constraint.

Why C is correct: Late fusion operates entirely on the output distributions of already-trained models, requiring no modification to the specialist model weights. A lightweight aggregation layer or weighted vote is trained on validation predictions alone, satisfying the cost constraint whilst combining all three modality signals.

Why D is wrong: Knowledge distillation requires training a new student model end-to-end, which is a form of retraining that incurs comparable or greater cost than fine-tuning the specialists. This does not satisfy the cost-prohibitive retraining constraint.

A transcription product feeds raw ASR output into a downstream summariser, but the summaries read poorly because the ASR text arrives as a single lowercase run of words with no full stops or commas. The acoustic transcription itself is accurate. Which capability should be added to the ASR pipeline to fix the readability of the transcript?

• ABeam search decoding with a wider beam so that more candidate transcriptions are considered per frame.
• BVoice activity detection so that the audio is segmented into utterances before transcription begins.
• CA punctuation and capitalisation post-processing model applied to the recognised token sequence.check_circle Correct
• DSpeaker diarisation so that each segment of the transcript is labelled with which speaker produced it.

Why A is wrong: A wider beam can improve which words are chosen, which is why it tempts, but the words are already accurate and beam search does not insert sentence punctuation or capitalisation, so it leaves the readability problem unchanged.

Why B is wrong: Voice activity detection usefully splits audio into speech segments, so it seems related to structure, but it operates on the audio rather than the text and does not add the commas, full stops, and capitals the transcript lacks.

Why C is correct: Raw ASR output is typically unpunctuated and lowercase, so a dedicated punctuation and capitalisation model restores sentence boundaries and proper casing, directly producing the readable text the summariser needs.

Why D is wrong: Diarisation adds who-spoke-when labels and is valuable for multi-speaker logs, which makes it plausibly relevant, but it does not insert punctuation or capitalisation, so it does not solve the run-on lowercase text.