A developer is building a customer-support chatbot that must answer questions about a company's product catalogue, which is updated weekly. The catalogue contains roughly 200,000 tokens of text. Which design consideration most directly determines whether a single foundation model call can process the entire catalogue without retrieval augmentation?
- AThe model's training data cutoff date, because any catalogue content newer than the cutoff will be ignored by the model during inference.
- BThe model's output token limit, because generating long catalogue summaries requires a high output ceiling to avoid truncation.
- CThe model's context window size, because the full catalogue must fit within the input limit for a single inference call. Correct
- DThe model's temperature setting, because a lower temperature ensures the model attends to all catalogue entries rather than sampling selectively.
Why A is wrong: Training cutoff affects factual knowledge baked into weights, not the ability to process input tokens supplied at inference time. Providing the catalogue as context bypasses the cutoff problem entirely, so this is not the limiting factor here.
Why B is wrong: Output token limits affect response length, not input capacity. The question asks about processing the catalogue as input, so output limits are irrelevant to this constraint.
Why C is correct: Context window defines the maximum tokens a model can accept per call. If the catalogue exceeds that limit, the model cannot process it in one shot, making retrieval augmentation necessary.
Why D is wrong: Temperature controls output randomness and has no bearing on how many input tokens the model can accept. It is a tempting distractor because 'attending to all entries' sounds plausible, but attention and context length are separate concepts.