CCAR-F - Prompt Engineering & Structured Output - Section 4.4

Implement validation, retry, and feedback loops for extraction quality.

Retry with error feedback appends the specific validation failures to the follow-up request along with the original document and the failed extraction, guiding self-correction. Retry has a hard limit: it cannot recover information that is simply absent from the source. Candidates should design self-correction flows that extract a calculated total alongside a stated total to expose discrepancies, and should track which constructs trigger findings to analyse dismissal patterns systematically.

retry with error feedbacklimits of retry when information is absentsemantic versus schema validation errorsdetected_pattern trackingconflict detection flags

Practice question for this objective

Free samplePrompt Engineering & Structured Outputmedium

A validation step in the Ledger Intake pipeline rejects any extract_invoice call that fails the schema, then re-sends the same document together with the validator error message appended to the request. Across one week this recovers 96 percent of failures in which invoice_total arrived as a string carrying a currency symbol, but only 4 percent of failures in which invoice_number is required and the page has been cropped so the header no longer appears. What best explains that split?

  • AThe cropped cases fail because the retry re-sends the document without the validator message attached, leaving the model with no more information than it had on the first attempt.
  • BThe cropped cases fail because repeated requests are served from a response cache, so an identical document keeps returning the identical invalid arguments no matter what feedback is appended.
  • CCorrective feedback tells the model how to reshape a value it already holds, which repairs a badly formatted number, but no feedback can recover an identifier the cropped page does not contain. Correct
  • DThe cropped cases fail because a cropped scan pushes the request past the model's attention budget, so raising the context limit on those requests restores the missing header field.
Retrying with validator feedback repairs malformed values but cannot recover information that is absent from the source document. A schema rejection carries a description of what was wrong with the arguments, which is actionable when the underlying value exists and was merely represented badly, such as a number emitted as a string with a symbol attached. When the required field simply is not present in the source, the retry loop has nothing new to draw on and each attempt either fails again or fabricates a value. Absent source data needs a different handling path, such as routing the document for rescanning or recording the field as unavailable.

Why A is wrong: Tempting because a retry that drops its corrective feedback is a genuine implementation bug worth checking for. The described pipeline does attach the error message, and the currency symbol cases recover precisely because that feedback arrives.

Why B is wrong: Tempting because caching does make retries look futile and is a real cost optimisation in production pipelines. A cache would suppress recovery on both failure classes equally, which contradicts the 96 percent recovery on the formatting faults.

Why C is correct: Correct. The two failure classes differ in what the retry can supply. A formatting fault is fully recoverable because the information is present and only the representation is wrong, whereas a missing header removes the information itself, so further attempts can only produce a guess.

Why D is wrong: Tempting because attention dilution on long inputs is a real limitation and cropping sounds like a size problem. Cropping removes content rather than adding it, and no context setting can reinstate pixels that were never sent.

See more CCAR-F practice questions, answers explained.

More in this domain

Back to all Prompt Engineering & Structured Output objectives, or the CCAR-F cert hub.

Examworthy is not affiliated with or endorsed by Anthropic. Original, blueprint-aligned practice material only.