NCA-GENL - Data Analysis and Visualization (14% of the exam) - Section 4.5
Use transformer models for text classification and named-entity recognition.
Use pre-trained transformer models for text classification tasks such as sentiment analysis and topic labelling, and for named-entity recognition (NER) tasks that tag spans as people, locations, or organisations. Distinguish the fine-tuning objectives for sequence-level classification versus token-level NER, and evaluate results with precision, recall, and F1.
Text classificationNER
Practice question for this objective
lock_openFree sampleData Analysis and Visualizationmedium
A transformer-based text classifier and a transformer-based named-entity recogniser both sit on top of the same pre-trained encoder, yet they attach different prediction heads. Conceptually, what most fundamentally distinguishes the two task formulations at the output stage?
AText classification requires a softmax output layer, whereas NER must always use a sigmoid output layer because entities can overlap within a sentence.
BText classification reads only the final token's hidden state, whereas NER reads only the first token's hidden state.
CText classification freezes the encoder while NER fine-tunes it, which is why their heads differ.
DText classification assigns one label to the whole sequence using a single pooled representation, whereas NER assigns a label to every token using each token's own contextual representation.check_circle Correct
Distinguish sequence-level text classification from token-level NER by the granularity of their prediction heads on a shared encoder. Text classification maps the entire input to one label through a pooled or special-token vector, while NER is sequence labelling that produces one tag per token from each position's contextual embedding; this output granularity is the core structural distinction.
Why A is wrong: Tempting because activation choice does vary by task, but it is wrong: standard single-type NER uses a per-token softmax over the tag set, and sigmoid is tied to multi-label problems, not to NER inherently.
Why B is wrong: Tempting because both tasks do select specific positions, but it is wrong: classification conventionally pools or uses the prepended classification token, and NER uses every token's state, not just the first.
Why C is wrong: Tempting because freezing strategies exist, but it is wrong: the head difference comes from the granularity of the prediction, not from whether the encoder is frozen, and either task can freeze or fine-tune the encoder.
Why D is correct: Correct: classification is a sequence-level task that consumes one aggregated vector, while NER is a token-level (sequence-labelling) task that emits a label per token position, which is the defining structural difference.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENL bank for this domain.
The encoder assigns every token a fixed embedding from a lookup table, which guarantees that identical words always receive identical entity tags.
errorWhy it is wrong: Tempting because lookup embeddings exist as an input layer, but it is wrong: the strength of the transformer is that its output representations are contextual, not fixed, and forcing identical tags for identical words would hurt NER.
Increase the model's hidden size and add two extra transformer layers to improve representational capacity
errorWhy it is wrong: Adding capacity helps when the model is under-fitting the data, not when the training signal is dominated by majority classes. A larger model trained with unweighted loss on an imbalanced dataset will still predict the majority class for ambiguous inputs, leaving rare-class recall unchanged.
The mean of the token embeddings read from the input embedding layer, taken before any transformer block has processed the sequence
errorWhy it is wrong: Averaging the raw input embeddings is tempting because it summarises every token, but those vectors carry no contextual information from self-attention, so the head receives a far weaker signal than a contextualised representation and classification accuracy suffers.