CCAR-F - Prompt Engineering & Structured Output (20% of the exam) - Section 4.2
Apply few-shot prompting to improve output consistency and quality.
Few-shot examples are the most effective route to consistently formatted, actionable output when detailed instructions alone produce variance. They are how ambiguous-case handling is demonstrated, and they let the model generalise judgement to novel patterns instead of matching only the cases listed. Candidates should write two to four targeted examples that show the reasoning for choosing one action over a plausible alternative, and use them to cut hallucination in extraction from varied document structures.
few-shot examplesambiguous-case demonstrationgeneralisation to novel patternsoutput format demonstrationhallucination reduction in extraction
The Ledgerline team runs a document extraction service that asks the model to return supplier name, invoice number, invoice date and total as a JSON object, then validates that object against a JSON schema before it reaches the accounts payable system. Parse failures sat at 4 percent. An engineer adds three worked examples to the prompt, each pairing a short invoice with the exactly formatted JSON object it should produce, and parse failures fall to 0.6 percent across the next 20,000 documents. The architect has to record what the example set now guarantees about output structure. Which statement is correct?
AThe examples raise the probability of the demonstrated shape without bounding it, so schema validation remains the only stage in the pipeline where a malformed object is actually rejected before the downstream system sees it.check_circle Correct
BThe examples make the output shape deterministic for any document resembling the three worked cases, so schema validation is now redundant for that class of document and can be relaxed to a sampled check.
CThe examples constrain decoding to the demonstrated shape, so a malformed object can now arise only from an illegible source document rather than from the generation step itself.
DThe examples have no bearing on structure, and the fall in parse failures is attributable to the schema validation stage rejecting malformed objects and re-requesting them from the model.
Few-shot examples raise the likelihood of a demonstrated output format; only validation of the produced object enforces it. Worked examples condition the model towards the demonstrated structure, which reliably lowers malformed output, but the model still samples each response, so no example count converts a probability into a guarantee. The deterministic guarantee lives in the schema validation stage that inspects the object after generation and rejects it, which is why that stage must stay in place even after conformance improves sharply.
Why A is correct: Correct: few-shot demonstration shifts the output distribution towards the shown format, which is why the rate fell, while the deterministic guarantee still comes from validating the produced object and refusing it on failure.
Why B is wrong: Tempting because the measured improvement is large and consistent, but a demonstrated shape is still produced by sampling, so a lower failure rate is not a bound, and relaxing validation removes the one stage that actually rejects a malformed object.
Why C is wrong: Plausible because few-shot examples do strongly influence formatting, but examples are conditioning rather than a decoding constraint, so the generation step retains its own residual failure rate independently of document quality.
Why D is wrong: Tempting because validation genuinely does catch malformed output, but validation was already running before the change at a 4 percent failure rate, so it cannot explain an improvement that followed the addition of the examples.
Exam traps in Prompt Engineering & Structured Output
Answers that look right on this material and are not. Each one is a distractor from a different question in the CCAR-F bank for this domain.
The schema marks total as nullable, and a nullable field is treated as lower priority during generation, so the model fills it opportunistically from the nearest currency figure printed on the page.
errorWhy it is wrong: Tempting because the fabricated value often does resemble a nearby figure, but nullability in the schema is a validation property applied after generation and carries no ranking of fields during generation.
Approve the proposal, on the basis that doubling the number of demonstrations increases the weight the model places on the demonstrated field ordering.
errorWhy it is wrong: Tempting because example count and output consistency do correlate up to a point. It is wrong because the additional examples carry no information about the eleven templates that are failing, so they deepen the fit to two page layouts and can push new-template accuracy lower.
Move the extraction work to the Message Batches API and return the fields to the user once the submitted batch completes within its asynchronous window.
errorWhy it is wrong: Tempting because batch processing genuinely reduces the cost of high-volume extraction. It is wrong because this workflow blocks a user waiting at an upload screen, and an asynchronous window measured in hours cannot satisfy a three second interactive requirement.