Security and Safety

An internal operations assistant has a tool that permanently deletes an archived customer record, and deletion cannot be undone. Policy states that a named data owner must approve every deletion. An audit of one quarter found nine deletions carried out with no approval recorded, despite a system prompt telling the assistant to confirm with the owner first. The assistant must keep its other archive tools. Which design meets the policy?

• AKeep the deletion tool and require the assistant to state the owner's name and the reason in the tool arguments, so that every deletion carries an attributable justification in the call record.
• BSplit the deletion tool so that the assistant can only write a pending deletion request, and have a separate service perform the deletion after the named owner approves the request through their own interface.check_circle Correct
• CAdd a second system prompt paragraph, written in stronger terms, instructing the assistant to obtain approval and to refuse the deletion outright if the owner has not replied within the conversation.
• DLog every deletion call to the audit store and review the log each week, raising any deletion that lacks a recorded approval with the team that operates the assistant.

Why A is wrong: Richer arguments give a useful audit trail, which is why teams reach for this, but the model supplies those arguments itself and the deletion still executes on its say-so, so an unapproved deletion remains possible.

Why B is correct: The irreversible step is removed from the assistant's reach entirely, so the only path to a deletion runs through a recorded human approval, while the remaining archive tools are unaffected.

Why C is wrong: This reads like a tightening of the existing control, but it is the same class of control that already failed nine times, and an instruction cannot prevent the tool from being invoked.

Why D is wrong: Logging and review are genuinely required for accountability, but they detect an unapproved deletion after the record has gone, and the policy demands approval before an irreversible action.

A support assistant answers questions about open tickets and can call a tool that closes a ticket. Customers write the ticket text themselves. Over 5,000 live conversations, an audit found 23 tickets closed immediately after a customer message containing wording addressed to the assistant. The team must reduce that count to zero while the assistant keeps reading customer-written ticket text. What is the correct change?

• AExplain in the system prompt that customer ticket text is untrusted, and require the assistant to confirm in its reply that a closure request came from an adviser before it calls the closing tool.
• BStrip formatting, quotation marks and imperative sentences from customer ticket text before it is placed in the context, so that instruction-shaped wording is removed prior to any tool call.
• CLower the sampling temperature for conversations that include customer-written ticket text, so the assistant follows its own instructions more consistently and closes fewer tickets in error.
• DMake ticket closure an application-side action that the assistant can only recommend, with the adviser's own authenticated click performing the close, and leave the assistant holding read-only ticket tools.check_circle Correct

Why A is wrong: Tempting because a self-declared check reads like an approval gate and can be added today. The confirmation is produced by the same turn the injected text is influencing, so the assistant can assert the check passed and close the ticket anyway.

Why B is wrong: Tempting because sanitising input is standard practice against injection attacks in other settings. Natural language carries intent without any of those markers, so the filter mangles genuine ticket text while an ordinary polite sentence still conveys the instruction.

Why C is wrong: Tempting because the failure looks like inconsistency, and temperature is the obvious consistency lever to reach for. Temperature affects token sampling, not whether an instruction inside the context is honoured, so it cannot bring a security count to zero.

Why D is correct: Correct because the mutating step moves outside the model's reach and is bound to an authenticated adviser action. Injected text can then produce only a recommendation, which drives the audited count to zero rather than merely lowering it.

An internal assistant answers staff questions about HR documents through a tool that reads a document store. The tool handler authenticates to that store with one service account able to read every document, and the system prompt tells the assistant to answer only from documents the asking employee is entitled to see. An audit found staff receiving contents of documents restricted to their managers. The requirement is that a tool call can reach only the documents the requesting employee is entitled to. Which change meets it?

• AHave the tool handler query the store under the requesting employee's own authenticated identity, taken from the session the application already established, so the store applies that employee's entitlements to the query.check_circle Correct
• BInsert the requesting employee's name and grade into the system prompt on every request, and instruct the assistant to discard any retrieved document that employee is not entitled to see.
• CHave the model supply the requesting employee's identifier as an argument to the tool, and have the handler filter the store query by whatever identifier arrives in that argument.
• DReduce the service account's permissions to the least sensitive document class, so no manager-restricted document can be returned to the assistant under any circumstances.

Why A is correct: Correct: the tool call then acts with the end user's authority rather than the service account's, so the document store enforces the same entitlements it would apply if the employee searched it directly, and no restricted content reaches the context.

Why B is wrong: Tempting because the assistant then has the information needed to filter, but the restricted content has already been read into the context by a service account that could reach it, and an instruction to discard it is guidance the model may not follow under conversational pressure.

Why C is wrong: Tempting because authorisation now appears to happen in code rather than in the prompt, but a tool argument is produced by the model from conversation text a user can influence, so it is an unauthenticated claim of identity rather than a verified one.

Why D is wrong: Tempting because it does stop the audited disclosure immediately, but it collapses every user to one permission level and removes legitimate manager access, so the assistant no longer satisfies the entitlement requirement it was built to serve.

An insurer's Claude-based assistant decides whether a submitted claim may be settled automatically by calling an internal fraud-scoring service. During a 25 minute outage of that service the assistant continued to settle claims, treating the missing score as a low-risk result, and the insurer later reversed 40 settlements. The insurer requires that a dependency failure never widen what the assistant is permitted to do. Which definition of safe failure should the team implement for this tool?

• AOn a failed scoring call, retry the service ten times with a fixed one second gap and, if every attempt fails, proceed with the settlement while writing a warning line into the application log for later review.
• BOn a failed scoring call, have the integration layer refuse the settlement action, return an explicit unavailable result to the assistant, and route the claim to the human queue until scoring is confirmed healthy.check_circle Correct
• COn a failed scoring call, substitute the median fraud score observed across the previous month so the assistant has a value to reason over, and mark the claim for a sampled quality review after settlement.
• DOn a failed scoring call, remove the settlement tool from the request and instruct the assistant in the system prompt to explain that automatic settlement is unavailable until scoring returns.

Why A is wrong: Retrying a transient failure is reasonable and logging the outcome preserves a record, but proceeding after exhausted retries still settles the claim without a score, which is the exact widening of permission the insurer forbids.

Why B is correct: Failing closed in the integration layer means an absent score can never be read as a favourable one, and routing to the human queue keeps the claim moving through a path that already has the authority to decide it.

Why C is wrong: A statistical stand-in keeps the pipeline moving and feels defensible, but an invented score is not evidence about this claim, and a review after settlement corrects the outcome only once money has already moved.

Why D is wrong: Withdrawing the tool is closer to correct and would block the action, but pairing it with a prompt instruction as the stated control leaves the behaviour resting partly on wording, and it gives the claim no route to a human while the outage lasts.