A retail bank uses Claude to extract declared income from payslips that applicants upload during a mortgage application. After the bank launched a mobile upload route alongside its existing web form, the share of extractions that underwriters correct rose from 2 to 9 percent, and 85 percent of the corrected cases arrived through the mobile route. The prompt, model and output schema were not changed in that release, and every extraction still passes schema validation. Where should the architect look first?
- AThe model tier: whether a more capable model reads low quality phone photographs more accurately than the current model does.
- BThe input stage: what the mobile route actually submits, such as page count and image legibility, compared with web uploads. Correct
- CThe output schema: whether its field types are loose enough to let a plausible but incorrect income value pass validation.
- DThe prompt: whether its extraction instructions cover the newer payslip layouts that some employers have started to issue.
Why A is wrong: This is tempting because a stronger model may cope better with poor images. It is wrong as a first step because the model did not change and the failure followed a change to the input path; upgrading the model compensates for bad input instead of finding out what the new route sends, and cannot recover a page that never arrived.
Why B is correct: This is correct because the only thing that changed is how documents arrive, and the errors concentrate in the new route. Comparing what the mobile route delivers, for example missing pages or blurred photographs, with web uploads tests the most likely cause directly, and points to input validation that rejects or flags incomplete documents before the model call.
Why C is wrong: This is tempting because every extraction passes schema validation and yet some are wrong. It is wrong as the first place to look because a schema only checks shape and type, it did not change, and it cannot explain why the errors cluster in mobile uploads; tightening it would not address the cause.
Why D is wrong: This is tempting because unfamiliar document layouts are a common cause of extraction errors. It is wrong because new employer layouts would reach the bank through both upload routes equally, whereas 85 percent of the corrected cases came through the mobile route, which points at the channel rather than the document design.