GH-300 - Understand GitHub Copilot data and architecture (13% of the exam) - Section 3.3

Describe the limitations of large language models and GitHub Copilot.

Describe the limitations of the large language models behind GitHub Copilot - knowledge cutoffs, bounded context windows, no genuine understanding, and the chance of confident but wrong output - and what those limitations mean for how much a developer should rely on a suggestion.

LLM limitationsContext windowKnowledge cutoffConfident errors

Practice question for this objective

Free sampleUnderstand GitHub Copilot data and architectureeasy

A developer asks Copilot Chat to confirm the syntax for a configuration flag that was deprecated and replaced after the model's training data was collected. Copilot returns the old flag in a fluent, confident answer. Which limitation of large language models best explains this behaviour?

AThe content exclusion settings have removed the configuration file from the model's available context window.
BThe model has a knowledge cutoff, so it cannot know about changes made after its training data was gathered. Correct
CChat answered from earlier turns in the same conversation, reusing a cached reply rather than consulting the model for the current question.
DThe Model Context Protocol blocked the newer documentation from reaching the underlying language model.

Recognise that an LLM knowledge cutoff means information published after training is unknown, producing confident but outdated answers. Large language models learn from data collected up to a fixed point called the knowledge cutoff. Anything released after that date, such as a renamed flag, is outside the model's knowledge, so it answers fluently with the stale information it does hold.

Why A is wrong: Content exclusions stop named repository files being used as context; they do not cause the model to return outdated framework syntax, so this misattributes the cause.

Why B is correct: A large language model only knows patterns present up to its training cutoff, so changes published afterwards are unknown and it confidently returns the older syntax.

Why C is wrong: Each Chat turn is sent to the model for inference and is not served from a cached earlier reply, so conversation caching does not explain syntax that is stale because of training age.

Why D is wrong: Model Context Protocol connects external tools and data sources rather than blocking documentation, so it does not explain a stale answer caused by training age.

See more GH-300 practice questions, answers explained.

Exam traps in Understand GitHub Copilot data and architecture

Answers that look right on this material and are not. Each one is a distractor from a different question in the GH-300 bank for this domain.

Duplication detection suppressed the genuine framework documentation because it matched publicly available code too closely.
Why it is wrong: Duplication detection blocks suggestions matching public code; it does not suppress documentation or cause an invented interface, so it misattributes the cause.
The model only asserts facts it has confirmed against current documentation, so the claim can be quoted without any further checking.
Why it is wrong: The model does not confirm facts against current documentation before asserting them, so treating a confident claim as verified is the mistake to avoid.
Copilot only ever suggests code that has been verified against the installed dependencies in the project.
Why it is wrong: Copilot predicts plausible code from patterns and does not verify suggestions against installed dependencies, so this overstates the guarantees it provides.

Examworthy is not affiliated with or endorsed by GitHub. Original, blueprint-aligned practice material only.