A retail bank wants Claude to draft replies to customer complaints. Discovery has established that the team handles about 4,000 complaints a month, that the complaints policy library holds 2,300 documents which the compliance team revises weekly, and that the regulator requires each reply to cite the policy clause, and its version, that was in force on the date of the complaint. The sponsor proposes fine-tuning a model on two years of approved replies. What should the architect recommend?
- AFine-tune a model on the two years of approved replies so policy positions and house style are carried in the model weights.
- BLoad the whole policy library into the context window on each request so that retrieval cannot miss a relevant policy clause.
- CPrompt the model with a summary of current policy positions, and have a reviewer add the clause citations before each reply is sent.
- DRetrieve clauses from a versioned policy index filtered by complaint date, and have each draft cite the retrieved clause and version. Correct
Why A is wrong: This is tempting because the sponsor proposed it and two years of approved replies look like ideal training data for tone and policy. It is wrong because a weekly revision cycle would leave the weights stale within days, and a fine-tuned model cannot point to which clause version it relied on, so the citation requirement cannot be met.
Why B is wrong: This is tempting because it removes retrieval misses as a failure mode. It is wrong because 2,300 documents on each of 4,000 requests a month is costly and slow, the model still has to pick the version in force on the complaint date from a mass of competing versions, and long-context stuffing is the over-engineered answer where targeted retrieval fits.
Why C is wrong: This is tempting because it keeps a human in the loop on a regulated output and avoids building an index. It is wrong because a summary of current positions cannot reflect the version in force on an older complaint date, and moving the citation duty to a manual step is a compensating control that adds cost to every reply where a structural fix exists.
Why D is correct: Correct. The discovered facts (weekly revisions, a large library, and a date-specific citation duty) all point to retrieval over a versioned index: updates reach answers as soon as they are indexed, the date filter selects the version in force, and the retrieved clause gives the draft something concrete to cite.