NCA-GENL - Core Machine Learning and AI Knowledge - Section 1.3

Explain self-supervision improvements in BERT, Megatron, and other LLM variants.

Explain how self-supervision eliminates the need for manually labelled data by deriving training signals from the input itself, as seen in BERT's masked language modelling objective. Distinguish how Megatron extends this approach through tensor and pipeline parallelism to train models at a scale that single-GPU training cannot reach.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.