CCAR-F - Prompt Engineering & Structured Output (20% of the exam) - Section 4.3
Enforce structured output using tool use and JSON schemas.
Tool use with a JSON schema is the reliable route to schema-compliant output and removes syntax errors, but it does not prevent semantic errors such as line items that do not sum to the stated total or values placed in the wrong field. Candidates should design required versus optional fields, mark fields nullable so the model returns null instead of fabricating a value, add enum members such as unclear and an other plus detail pattern, and set tool_choice to force the right extraction tool.
tool_use with JSON schemasyntax errors versus semantic errorsnullable and optional fieldsenum with other plus detailtool_choice forced selection
A document extraction service defines a single tool, extract_invoice, and reads the structured fields from the tool_use block of the response. Production logs over one week show that in 3.1 percent of requests the model returns only prose, usually a clarifying question about a poorly scanned line, with stop_reason end_turn and no tool_use block present, and the downstream consumer raises an unhandled error each time. Compliance requires that every extraction request return a structured payload. Which change most effectively meets that requirement?
AAdd a line in capitals to the system prompt instructing the model to reply with the structured payload and to ask no clarifying questions of any kind.
BCatch the parse failure in the consumer and reissue the identical request up to three times, treating the run as failed only if all attempts return prose.
CLeave tool_choice at auto and supply four few shot exchanges that show a structured tool call made against a badly scanned invoice.
DSet tool_choice to the named tool extract_invoice on the extraction request so the response is required to contain a call to that tool.check_circle Correct
When a structured response is mandatory, force it with tool_choice set to the named tool rather than relying on prompt instructions. With tool_choice at auto the model decides whether to call a tool, so a prose reply with stop_reason end_turn is a legitimate outcome and no wording in the system prompt can remove it. Setting tool_choice to the named tool changes the contract at the API layer, so the response must contain a call to that tool and the consumer always receives a schema validated payload.
Why A is wrong: Tempting because emphatic instructions usually reduce the behaviour, but prompt guidance is probabilistic and a compliance requirement stated as every request needs a guarantee the API itself enforces.
Why B is wrong: Reasonable resilience engineering, but it adds latency and cost on every affected document while still leaving a residual failure rate that the compliance wording does not permit.
Why C is wrong: Few shot examples do shift behaviour and are cheap to add, but auto keeps the prose reply available to the model, so the guarantee the requirement asks for is still missing.
Why D is correct: Correct because naming the tool in tool_choice forces the model to emit a call to it, which removes the prose only response as a possible outcome rather than merely discouraging it.
Exam traps in Prompt Engineering & Structured Output
Answers that look right on this material and are not. Each one is a distractor from a different question in the CCAR-F bank for this domain.
Retry every low quality scan twice at a lower temperature and accept the total only when the two independent runs return exactly the same figure.
errorWhy it is wrong: Tempting because agreement across runs usually raises confidence, but retries cannot recover information that is not present in the image, and two guesses can agree on the same wrong figure.
Harden the regular expression so that it tracks brace depth and ignores any brace that falls inside a quoted string within the captured text.
errorWhy it is wrong: Tempting because it targets the exact observed trigger, but it rebuilds a JSON parser out of a pattern and stays fragile against escapes and other characters the contracts will eventually contain.
That each argument was read from the region of the page named by its schema field, because validation runs against the extracted spans rather than against the finished arguments.
errorWhy it is wrong: Tempting because schema fields carry meaningful names such as invoice_total, which invites the assumption that the name binds the value to a location on the page. Validation never sees the document, only the arguments the model produced, so it cannot check provenance.