NCA-GENL - Software Development - Section 2.3

Implement transformer-based models for different NLP applications.

Implement transformer-based models for NLP tasks including text classification, summarisation, question answering, and machine translation, selecting the correct pre-trained checkpoint and tokeniser for each. Recognise the differences between encoder-only, decoder-only, and encoder-decoder variants, and apply the right architecture to the task at hand.

Practice question for this objective

Free sampleSoftware Developmentmedium

A team must select a transformer architecture family for two separate applications: a sentiment classifier that produces one label per document, and an autoregressive assistant that generates free-form text token by token. They want each application matched to the architecture whose attention pattern suits it. Which architecture-to-task pairings are correct? Select TWO.

  • AAn encoder-only model with bidirectional self-attention fits the single-label document classifier, since each token can attend to the full left and right context before pooling. Correct
  • BA decoder-only model with causal masked self-attention fits the autoregressive assistant, since each position attends only to earlier tokens to predict the next one. Correct
  • CAn encoder-only model with causal masking is the natural fit for free-form text generation because masking future tokens is unique to encoder architectures.
  • DA decoder-only model with full bidirectional attention over the entire sequence is the correct choice for single-label document classification.
  • EBoth applications are best served by an encoder-decoder model because the cross-attention bridge is mandatory for any task that emits a class label or a token.
Bidirectional encoder-only models suit single-label classification while causal decoder-only models suit autoregressive next-token generation. Encoder-only transformers attend bidirectionally so each token sees the full context, giving representations ideal for a pooled classification head, whereas decoder-only transformers apply causal masking so each position depends only on prior tokens, which is precisely the constraint that next-token generation needs.

Why A is correct: Correct because bidirectional attention lets every token see the whole document, producing rich representations well suited to a single-label classification head.

Why B is correct: Correct because causal masking prevents a position from seeing future tokens, which is exactly what next-token generation requires.

Why C is wrong: Tempting by mixing terms, but wrong: encoders are bidirectional by default, and causal masking for generation is the hallmark of decoder-only models.

Why D is wrong: Wrong on two counts: decoder-only models use causal not bidirectional attention, and they are not the conventional pick for single-label classification.

Why E is wrong: Plausible since encoder-decoders are versatile, but wrong: cross-attention suits sequence-to-sequence mapping, not single-label classification or pure next-token generation.

See more NCA-GENL practice questions, answers explained.

More in this domain

Back to all Software Development objectives, or the NCA-GENL cert hub.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.