CCAR-F - Prompt Engineering & Structured Output (20% of the exam) - Section 4.4
Implement validation, retry, and feedback loops for extraction quality.
Retry with error feedback appends the specific validation failures to the follow-up request along with the original document and the failed extraction, guiding self-correction. Retry has a hard limit: it cannot recover information that is simply absent from the source. Candidates should design self-correction flows that extract a calculated total alongside a stated total to expose discrepancies, and should track which constructs trigger findings to analyse dismissal patterns systematically.
retry with error feedbacklimits of retry when information is absentsemantic versus schema validation errorsdetected_pattern trackingconflict detection flags
A validation step in the Ledger Intake pipeline rejects any extract_invoice call that fails the schema, then re-sends the same document together with the validator error message appended to the request. Across one week this recovers 96 percent of failures in which invoice_total arrived as a string carrying a currency symbol, but only 4 percent of failures in which invoice_number is required and the page has been cropped so the header no longer appears. What best explains that split?
AThe cropped cases fail because the retry re-sends the document without the validator message attached, leaving the model with no more information than it had on the first attempt.
BThe cropped cases fail because repeated requests are served from a response cache, so an identical document keeps returning the identical invalid arguments no matter what feedback is appended.
CCorrective feedback tells the model how to reshape a value it already holds, which repairs a badly formatted number, but no feedback can recover an identifier the cropped page does not contain.check_circle Correct
DThe cropped cases fail because a cropped scan pushes the request past the model's attention budget, so raising the context limit on those requests restores the missing header field.
Retrying with validator feedback repairs malformed values but cannot recover information that is absent from the source document. A schema rejection carries a description of what was wrong with the arguments, which is actionable when the underlying value exists and was merely represented badly, such as a number emitted as a string with a symbol attached. When the required field simply is not present in the source, the retry loop has nothing new to draw on and each attempt either fails again or fabricates a value. Absent source data needs a different handling path, such as routing the document for rescanning or recording the field as unavailable.
Why A is wrong: Tempting because a retry that drops its corrective feedback is a genuine implementation bug worth checking for. The described pipeline does attach the error message, and the currency symbol cases recover precisely because that feedback arrives.
Why B is wrong: Tempting because caching does make retries look futile and is a real cost optimisation in production pipelines. A cache would suppress recovery on both failure classes equally, which contradicts the 96 percent recovery on the formatting faults.
Why C is correct: Correct. The two failure classes differ in what the retry can supply. A formatting fault is fully recoverable because the information is present and only the representation is wrong, whereas a missing header removes the information itself, so further attempts can only produce a guess.
Why D is wrong: Tempting because attention dilution on long inputs is a real limitation and cropping sounds like a size problem. Cropping removes content rather than adding it, and no context setting can reinstate pixels that were never sent.
Exam traps in Prompt Engineering & Structured Output
Answers that look right on this material and are not. Each one is a distractor from a different question in the CCAR-F bank for this domain.
The validator error arrives as a tool result rather than as a user turn, so the model treats it as untrusted output and rebuilds the field from its own priors instead of re-reading the page text.
errorWhy it is wrong: Tempting because tool results genuinely are a distinct content type and teams do worry about how error text is framed, but a tool result is ordinary model input here, and reframing it as a user turn would not conjure a purchase order that the cropped page never carried.
Raise the retry ceiling from three attempts to eight, so that a larger number of the failing invoices eventually yields a schema-valid object.
errorWhy it is wrong: More attempts is the intuitive response to a retry loop that is not converging, but each attempt resends an unchanged prompt and therefore samples the same distribution, so the extra attempts multiply cost while leaving the failure rate close to where it started.
It guarantees that a value the model could not read is returned as a null so the downstream system can quarantine the row, because a schema-typed field rejects an unreadable source value.
errorWhy it is wrong: Tempting because quarantining unreadable fields is the behaviour the team wants, but a schema constrains the shape of what is returned rather than detecting that the source was illegible, and nothing forces a null.