Model Selection and Optimization

An internal assistant answers questions about company travel policy from the model's own knowledge. An audit of 800 replies finds a measured 5 per cent that state policy details the company has never had, worded as confidently as the correct ones. An engineer proposes setting the sampling temperature to its lowest value, arguing that this will make the assistant stick to the facts. The team requires the invented details to stop. How should the proposal be assessed?

• AReject it, because temperature only shapes how the next token is chosen among candidates and does not supply the policy; the assistant should retrieve the approved policy text through a tool and answer from it.check_circle Correct
• BAccept it, because a low temperature makes the model select only the highest probability continuation, which is by construction the factually grounded one for a policy question.
• CAccept it, because temperature governs how much of the model's training data it is willing to draw on, and lowering it restricts the assistant to material it saw most often.
• DAccept it in part, because a low temperature removes the invented details while a moderate value would keep some of them, so the setting alone is enough to meet the requirement.

Why A is correct: Correct: the invented details come from the assistant having no source for the policy, and only giving it the real text at request time removes them.

Why B is wrong: Tempting because the most probable continuation often reads as the most confident, but probability under the model's training is not a measure of whether the company's policy says it.

Why C is wrong: Tempting because it sounds like a knowledge control, but temperature does not gate which training material is available; it only shapes selection among candidate next tokens.

Why D is wrong: Tempting because output does become more repeatable at a low temperature, but repeatable wrong answers still fail the requirement, so the setting alone does not meet it.

A support tool rejects any user message longer than 2,000 words before sending it to Claude, on the assumption that this keeps every request inside the model's input budget. Live traffic shows a measured 4 per cent of accepted messages still failing on length, and the failures cluster on pasted log extracts, code snippets and German product names. The team must stop the length failures without lowering the word limit for ordinary prose. What should the guard measure instead?

• AThe number of characters in the message, since a character count is a stable proxy for the model's input budget across every kind of text.
• BThe number of lines in the message, rejecting anything past a fixed line count, because pasted logs and code are the content that produced the measured failures.
• CThe number of tokens the message occupies once tokenised, counted with a tokeniser before the request is sent, and compared against the budget the request actually has.check_circle Correct
• DThe number of words in the message, keeping the existing limit but applying it after stripping whitespace and punctuation from the text first.

Why A is wrong: Character count is tempting because it is finer grained than words, but the ratio of characters to tokens still varies by script and by content, so a fixed character ceiling either rejects valid prose or lets dense text through.

Why B is wrong: Line count is tempting because the failing content is line oriented, but a single line can carry thousands of tokens and a long prose message can carry very few lines, so the guard would miss the cause.

Why C is correct: Correct: the model's input budget is denominated in tokens, and only counting tokens before sending measures the same quantity the request is checked against.

Why D is wrong: This is tempting because it looks like a refinement of the current guard, but the unit is still words, and the failing content is exactly the text where one word becomes many tokens.

An internal chat assistant keeps one conversation open for a whole working session. Finance reports that a request late in a session costs several times what the first one cost, even though the questions and the answers are about the same length throughout. The team has changed nothing else. What explains the rising cost?

• AThe model charges progressively more per token as a conversation lengthens, so a late turn is billed at a higher rate than an early one.
• BLater answers are internally longer because the model reasons over more material, so the output tokens billed rise steadily through a session.
• CThe service retains the conversation on the provider side and bills a storage charge for each turn kept available for the next request.
• DEvery request resends the whole conversation so far, so the input tokens billed per turn grow with the number of turns already exchanged.check_circle Correct

Why A is wrong: Tempting because the bill really does rise with conversation length, but the cause is the number of tokens sent rather than any change in the rate applied to each one.

Why B is wrong: Tempting because output is genuinely billed, but the stem states that answer length is unchanged, so output cannot account for a several-fold rise in cost.

Why C is wrong: Tempting because stateful chat products do exist, but the conversation is held by the calling application here, and there is no per-turn retention charge to explain the pattern.

Why D is correct: Correct: the API holds no state between calls, so the accumulated history is sent again on each turn and the input side of the bill grows as the transcript lengthens.

A browser-based writing assistant must show the first words of a reply within one second, and a full reply commonly takes eight seconds to generate. The team runs its own backend between the browser and the model, and the traffic pattern is one question from the user followed by one long reply. A developer proposes replacing the existing request and response call with a websocket connection held open for the whole session. What is the correct assessment?

• AThe websocket is required, because a plain HTTP response cannot deliver any part of a reply before the whole reply has been produced.
• BThe websocket is the better choice, because holding one connection open for the session lowers the total number of tokens the backend has to send for each reply.
• CThe websocket is not needed, because consuming the response incrementally and forwarding each chunk to the browser as it arrives already puts text on screen well inside the budget.check_circle Correct
• DNeither transport helps, because the one second budget can only be met by shortening the reply until the whole generation finishes inside one second.

Why A is wrong: It sounds right if streaming is assumed to need a persistent socket, but an HTTP response can be consumed incrementally, so the premise that nothing arrives early is false.

Why B is wrong: It is tempting because connection reuse does save some overhead, but the transport does not change how many tokens a reply contains, so the stated benefit does not exist.

Why C is correct: The traffic is one request and one long reply, so incremental consumption of the streamed response meets the first-token budget without a persistent bidirectional connection to operate.

Why D is wrong: Shortening the reply would indeed cut total time, but the budget is time to first text rather than time to completion, so it sacrifices answer quality for no reason.