Applications and Integration

A compliance team must summarise 60,000 archived support transcripts. Nobody waits on the result, the team has until the end of the week, and the stated requirement is the lowest cost per transcript. A developer proposes firing 60,000 concurrent realtime requests to finish sooner. Which approach best satisfies the stated requirement?

• AKeep the realtime requests but raise concurrency further, since finishing the job sooner is what reduces the total cost of the run.
• BKeep the realtime requests and lower max_tokens sharply on each one, so that each summary is cheaper to produce than it would otherwise be.
• CSubmit the transcripts through the Message Batches API, giving each request its own custom_id, and collect the results when the batch completes.check_circle Correct
• DKeep the realtime requests and enable prompt caching on the transcript body of each request, so that repeated content is charged at the cached rate.

Why A is wrong: It is tempting because wall-clock time and spend feel linked, but concurrency changes only how fast the same per-token charges accrue, so cost per transcript is unchanged.

Why B is wrong: It looks like a cost lever, but max_tokens is an upper bound that truncates a long answer rather than a discount, so it damages summaries without addressing the rate paid.

Why C is correct: Correct: the workload is large, uniform and latency tolerant, which is exactly the shape the batch path exists for, and it is charged at a lower rate than equivalent realtime calls.

Why D is wrong: Caching is a real cost control, but it pays off on a stable shared prefix, and each transcript here is unique content that no later request reuses.

A claims intake service must return a structured record for every submitted claim, and it exposes one tool that writes that record. Over 2,000 live submissions, 9 per cent of requests returned a prose reply with no tool call at all, and the downstream system received nothing for those claims. The requirement is that a tool call is produced on every request. Which change removes the measured failure?

• ARaise max_tokens on the request, so the model has enough room to emit the tool call rather than stopping after its prose preamble.
• BLower the temperature on the request, so the model behaves more deterministically and consistently selects the tool it has been given.
• CSet tool_choice so that the request requires a tool call, which makes the tool invocation part of the request contract rather than a decision the model makes each time.check_circle Correct
• DAdd a firmly worded system-prompt instruction stating that the tool must be called on every claim, and repeat the instruction at the end of the user message.

Why A is wrong: Truncation is a real cause of missing content and worth ruling out, but the failing responses end in prose rather than being cut short, so a larger output cap changes nothing about whether a tool is chosen.

Why B is wrong: Lower temperature does reduce variation in wording and is a common reflex for inconsistent behaviour, but it biases sampling rather than constraining the response shape, so a share of prose-only replies survives.

Why C is correct: tool_choice controls whether tool use is optional or required, so requiring it removes the model's discretion and guarantees the structured call the downstream system depends on.

Why D is wrong: Clear instructions do raise compliance and are cheap to try, but an instruction is guidance rather than a constraint, which leaves a residual failure rate instead of removing it.

A legal assistant sends a long, unchanging policy corpus at the head of every request, followed by the user's question. Spend per request is over the team's stated ceiling and the corpus cannot be shortened. An engineer enables prompt caching but places the cache breakpoint after the user's question. Requests stay expensive. What explains this and what should change?

• ACaching applies only to output tokens, so a request dominated by input can never benefit; the team should shorten its answers to bring spend under the ceiling.
• BCaching is a latency feature rather than a cost feature, so the team should accept the spend and instead measure the improvement in time to first token.
• CThe corpus must be resent as a tool result rather than as leading context, because only tool results are eligible to be marked as cacheable content.
• DThe cached segment must end before the varying content, so the breakpoint should sit after the policy corpus and before the user's question in every request.check_circle Correct

Why A is wrong: It sounds plausible because output tokens cost more per token, but caching operates on the input prefix, and shortening answers leaves the expensive corpus paid for in full every time.

Why B is wrong: Faster prefill is a real side effect, which makes this tempting, but the primary reason to cache a large stable prefix is that reused input is charged at a reduced rate.

Why C is wrong: It sounds like a mechanism-level fix, but there is no such restriction on where cacheable content sits, and moving the corpus does nothing about a breakpoint that follows varying text.

Why D is correct: Correct: a cache hit requires the marked prefix to be byte-identical across requests, which the corpus is and the question is not, so the breakpoint belongs between them.

A retail operations assistant can issue refunds through a tool that moves money. Policy states that a refund above 500 pounds must be approved by a named supervisor before it is issued, and an audit found four unapproved high-value refunds issued last quarter despite a system prompt instruction telling the assistant to seek approval first. The requirement is that no such refund can be issued without a recorded approval. Which design meets it?

• AStrengthen the system prompt wording, requiring the assistant to state the approving supervisor's name in its reply before it invokes the refund tool for any high-value case.
• BSplit the work so a supervising agent reviews each refund the worker agent proposes and passes on only those it judges to be within policy for the customer in question.
• CLog every refund the assistant issues with its amount and reasoning, and have a supervisor review that log each morning and reverse anything that breached the stated threshold.
• DHave the refund tool record the request as pending above the threshold and release the payment only after the application receives a supervisor approval, returning the pending state to the assistant.check_circle Correct

Why A is wrong: Tempting because clearer wording usually improves adherence, but an instruction is guidance the model can fail to follow, which is exactly what the four audited refunds already demonstrated.

Why B is wrong: Tempting because a second reviewer looks like an approval step, but the reviewer is another model rather than the named supervisor, so no human approval is recorded and the check stays probabilistic.

Why C is wrong: Tempting because it produces a genuine audit trail with human review, but the money has already moved by then, so it detects breaches instead of preventing them as the requirement demands.

Why D is correct: Correct because the threshold test and the pause live in application code the assistant cannot bypass, so the payment cannot leave the system until a recorded approval exists.