NCA-GENM - Performance Optimization (10% of the exam) - Section 6.1
Leverage transfer learning between models.
Explain how transfer learning reuses weights from a pre-trained model to reduce training time and data requirements on a new task. Choose between freezing the backbone and fine-tuning only the head, or full fine-tuning, based on dataset size and domain similarity.
Transfer learning is sometimes split into domain adaptation and task transfer. Which statement most accurately contrasts these two situations?
ADomain adaptation keeps the same task but shifts the input distribution, while task transfer keeps related inputs but targets a different task, often requiring a new output head.check_circle Correct
BDomain adaptation always requires a brand-new output head, while task transfer never changes the output head because the label set is unchanged.
CDomain adaptation and task transfer both assume the input distribution and the label space are identical, differing only in the size of the training set used.
DTask transfer changes the input domain while keeping the same labels, whereas domain adaptation changes the labels while keeping the same input domain.
Contrast domain adaptation (input-distribution shift, same task) with task transfer (different task, often a new head). Domain adaptation handles a shift in the input distribution while the task and labels stay fixed, so the head can often be reused. Task transfer reuses learned features for a different task whose label space differs, which generally requires fitting a new output head.
Why A is correct: This is correct: domain adaptation addresses a change in input distribution for the same task, whereas task transfer reuses learned representations for a different target task, which typically needs a new output head for the new label space.
Why B is wrong: This inverts the typical pattern: a changed task usually needs a new head because the label space differs, whereas a domain shift with the same task often reuses the head. The absolute always and never also overstate the rule, so this is wrong.
Why C is wrong: This is tempting as a tidy unification, but it erases the very distinction: if both input distribution and label space matched, there would be no domain shift or task change to speak of, so the premise is incorrect.
Why D is wrong: This is a plausible-sounding swap of the definitions, but it reverses them: a changed input distribution with the same task is domain adaptation, and a changed task is task transfer, so the mapping here is backwards.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENM bank for this domain.
It permanently quantises the entire model to 4-bit integers before training, and the cost saving comes solely from the reduced numerical precision of every weight.
errorWhy it is wrong: This confuses LoRA with quantisation. Quantisation can be combined with LoRA, but low-rank adaptation's saving comes from training few parameters, not from reducing the precision of all weights, so attributing it solely to quantisation is wrong.
Train the entire network from random initialisation on the 800 board images so the features are perfectly specialised to circuit boards.
errorWhy it is wrong: Tempting because purpose-built features sound ideal, but discarding the pretrained weights removes the transferable low-level features and a network of this size trained from scratch on 800 images overfits even more severely than fine-tuning did.
Freeze the pretrained encoder entirely and train only a randomly initialised classification head on the medical dataset.
errorWhy it is wrong: Freezing all layers (feature extraction) is cheap but sacrifices domain adaptation; the encoder never learns medical vocabulary patterns, which limits performance on specialised text.