When adapting a pre-trained transformer to a downstream task, practitioners distinguish feature extraction from full fine-tuning. Which statement most accurately contrasts these two transfer-learning strategies?
- AFeature extraction keeps the pre-trained backbone frozen and trains only a new head, whereas full fine-tuning updates the backbone weights along with the head. Correct
- BFeature extraction always produces higher downstream accuracy than full fine-tuning because frozen weights cannot drift away from their pre-trained values.
- CFull fine-tuning trains only the embedding layer while leaving every transformer block frozen, which is why it needs less data than feature extraction.
- DFeature extraction and full fine-tuning both update every weight in the network and differ only in the learning rate applied to the output head.
Why A is correct: Correct. Feature extraction uses the frozen backbone as a fixed feature encoder and trains only the task head; full fine-tuning lets gradients update the backbone too, adapting the representation itself.
Why B is wrong: Tempting because frozen weights avoid catastrophic forgetting, but full fine-tuning usually reaches higher accuracy when enough labelled data is available, so 'always' is incorrect.
Why C is wrong: Tempting because partial-layer training exists, but full fine-tuning updates the whole network, not just embeddings, and it generally needs more data, not less.
Why D is wrong: Tempting because learning-rate tuning matters, but feature extraction by definition freezes the backbone, so the claim that both update every weight is wrong.