NCA-GENL - Software Development (24% of the exam) - Section 2.5
Write application code for generative tasks and chatbot applications.
Write application code that calls a generative model API, constructs prompts, and handles streaming or batch responses for tasks such as text generation, summarisation, and chatbot dialogue. Understand how to manage conversation history, control generation parameters like temperature and top-p, and integrate safety filters into an end-to-end application.
Practice question for this objective
lock_openFree sampleSoftware Developmentmedium
A developer is building a chatbot that must remember the last five exchanges. When constructing the messages array for each new API call, which approach correctly implements a sliding-window conversation history?
AAppend every new user and assistant turn to the messages list indefinitely, relying on the model to ignore older turns automatically.
BKeep the system message at index 0, then retain only the most recent ten messages (five user and five assistant turns), dropping the oldest pair first when the limit is reached.check_circle Correct
CStore only the last user message and the last assistant reply, discarding all earlier turns, so the model always receives exactly two messages plus the system prompt.
DPlace all historical turns before the system message in the messages array, then append the current user message at the end to give the model full context.
Understand how to manage conversation history in a chatbot by maintaining a bounded, correctly ordered messages array for LLM API calls. LLM inference APIs expect a structured messages array where the system message occupies position 0, followed by alternating user and assistant turns. A sliding-window strategy retains a fixed number of recent turns and removes the oldest pair when the limit is exceeded, keeping the payload within the model's context window while preserving coherent multi-turn context.
Why A is wrong: Tempting because it is the simplest implementation and some models do tolerate long contexts, but context windows have fixed token limits; indefinite growth will eventually cause the API to reject the request or silently truncate early turns, breaking conversation continuity.
Why B is correct: The system message provides persistent instructions and must stay in position 0; retaining the last ten role-alternating messages gives exactly five exchanges while bounding token consumption. Dropping the oldest pair first preserves chronological coherence and prevents context overflow.
Why C is wrong: Tempting because it guarantees a small payload and zero overflow risk, but limiting history to one exchange means the model cannot resolve references to earlier turns, producing incoherent multi-turn dialogue despite the chatbot appearing to respond.
Why D is wrong: Tempting because ordering history before the current question feels intuitive, but most inference APIs require the system message to be the first element; placing it after history violates the expected schema and will typically return a validation error.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENL bank for this domain.
The temperature parameter is set too high, causing the model to generate increasingly random completions that drift away from the original persona over successive turns.
errorWhy it is wrong: Temperature affects output randomness but does not cause the model to ignore its system instructions. Even a high-temperature setting would not make a correctly constructed request drop the system persona; the issue described is a missing context window problem, not a sampling problem.
The server automatically recompiles every model into a TensorRT engine each time a load request is issued, regardless of the model's backend.
errorWhy it is wrong: Tempting because TensorRT optimises models, but loading does not force TensorRT conversion; backends load their own artefacts as provided.
Set max_tokens to a low value calculated to approximate where '###' would appear, trusting length truncation to cut off generation at the right point.
errorWhy it is wrong: Tempting because limiting max_tokens does bound output length, but the model cannot guarantee it will emit '###' at the exact token budget; the generation may be cut before the delimiter or continue past it if the estimate is off, making downstream parsing unreliable.