CCAR-P domain - 17% of the exam

Solution Design & Architecture

Solution Design & Architecture is 17% of the CCAR-P exam. These are the objectives it covers, each with practice questions, with every answer explained.

The domain in numbers

54
Practice questions
6
Objectives
37
Medium
17
Hard

Where people go wrong

  • Trusting a model's confidence as an accuracy signal

    Several wrong answers add a confidence gate: the model rates its own answer, and the score decides what a person sees or what skips review. A model's self-reported confidence is not a calibrated accuracy signal. Feedback citing a removed criterion can be stated with high confidence, and a reviewer reading an excerpt that omits an exclusion criterion has no reason to feel uncertain. Where a rule can detect risk, such as a complex episode flagged from the record, route on that rule or on a measured check instead.

    See it in a sample question
  • Upgrading the model to fix a fault upstream of it

    The wrong answers respond to a quality problem by moving to a more capable model. In these questions the fault sits elsewhere: evaluation labels copied from the old misroutes, a missing input validation stage, retrieval serving a superseded edition, or a handoff that dropped a flag. A larger tier given the same flawed labels, incomplete application, stale passages or lossy recap reproduces the same result. Locate the stage where the defect enters, then fix that stage.

    See it in a sample question
  • Choosing a pattern by flexibility, not by whether the steps are known

    The wrong answers pick a pattern by how capable it sounds. Where the steps are known in advance, as with prior authorisation or a fixed intake route, they keep an autonomous agent or add one, which costs predictability for no benefit; a fixed workflow or a code change fits. The reverse error appears when the path depends on what each query returns: a fixed chain cannot follow it, and an agent with read-only tools, a step budget and analyst review is the fit.

    See it in a sample question
  • Measuring an easy proxy instead of the sponsor's outcome

    The wrong answers pick a measure or a design that is easy to count and sits beside the sponsor's outcome: survey responses, conversation volume, containment rate, token spend, per-step drafting help. A contained conversation is not resolved if the citizen contacts the centre again within 14 days, and trimming 20 minutes of work per step leaves queue time untouched. Tie success to the stated outcome, against a baseline taken before launch, and count the costs that outcome includes.

    See it in a sample question

Objectives in this domain

What to study

The official documents this domain's practice questions cite most.

Sample questions from this domain

Free sampleSolution Design & Architecturemedium

A hospital pharmacy team wants to flag paediatric prescriptions whose dose falls outside the weight-based range in its formulary table. The clinical safety officer requires that the same prescription and weight produce the same result every time, and that each flag can be traced to a specific table row during audit. A product owner proposes asking Claude to compute each safe range. What should the architect recommend?

  • ACompute each range in deterministic code against the formulary table, and limit Claude to drafting the plain-language alert text Correct
  • BHave Claude compute each range at a temperature of zero, since removing sampling randomness makes the calculation repeatable for audit
  • CHave Claude compute each range and route every flagged prescription to a pharmacist, so a human checks each calculation before release
  • DHave Claude compute each range with a confidence score, and send only those scored below a set threshold to a pharmacist for review
Assign rule-based calculations with reproducibility and audit requirements to deterministic code, and reserve generative components for tasks that tolerate variation. The safety officer's requirements are identical outputs for identical inputs and traceability to a table row. Deterministic code satisfies both by design, whereas any generative calculation, however constrained or reviewed, only approximates them. Splitting the system into a deterministic dose check and a generative alert-wording step uses each component where it fits.

Why A is correct: A table lookup and arithmetic in code is reproducible by construction and every flag points to the row that produced it, meeting both stated requirements. Claude is kept for the generative part, wording the alert, where variation does no harm.

Why B is wrong: It is tempting because lowering temperature does make outputs more consistent. It is wrong because it does not guarantee identical results across calls, and a generated calculation still cannot be traced to a specific formulary row in the way the safety officer requires.

Why C is wrong: It is tempting because human review feels like the safe choice in a clinical setting. It is wrong because it is a compensating control on a component that should not be generative at all, it misses unflagged prescriptions where the model erred, and it still fails the reproducibility requirement.

Why D is wrong: It is tempting because it appears to focus human effort where the risk is. It is wrong because a model's self-reported confidence is not a reliable accuracy signal, so confidently wrong ranges pass unchecked, and the approach still fails reproducibility and traceability.

Free sampleSolution Design & Architecturemedium

A software company's support desk loses about two days on each ticket that reaches the wrong team, and 21 per cent of tickets are currently reassigned at least once. The team built a Claude classifier whose prompt describes each team's remit, and evaluated it on 2,000 historical tickets, labelling each with the team recorded when the ticket was first assigned. It scored 93 per cent on that set, yet after launch the reassignment rate is still 21 per cent, and the mix of incoming ticket categories matches the evaluation set. What is the most likely cause?

  • AIncoming tickets have drifted away from the historical set since launch, so the 93 per cent score no longer holds live.
  • BThe classifier uses a model tier too small to separate closely related teams, so a larger tier would cut reassignments.
  • CThe evaluation labels came from each ticket's first assignment, so the classifier was scored on reproducing the old misroutes. Correct
  • DThe routing prompt lacks a description of each team's remit, so the model is guessing between teams with similar names.
Check that evaluation labels measure the business outcome, because labels taken from a flawed process reward reproducing its errors. An evaluation is only as meaningful as its ground truth. The labels recorded the first team assigned, which is the decision the business already knows is wrong about a fifth of the time. High agreement with those labels therefore measures fidelity to the current process, not correct routing. Defining success against the outcome the business cares about, such as the team that resolved each ticket or the live reassignment rate, would have exposed the gap before launch.

Why A is wrong: Tempting because distribution drift is a common reason offline scores fail to hold in production. It is wrong because the stem states that the live category mix matches the evaluation set, which rules drift out as the main explanation.

Why B is wrong: Tempting because a more capable model often separates fine-grained classes better. It is wrong because the classifier already matches its labels 93 per cent of the time; a larger model would match the same flawed labels more closely and leave the reassignment rate where it is.

Why C is correct: Correct. The first-assigned team is the output of the routing process that produces the 21 per cent reassignment rate, so roughly a fifth of the labels are themselves wrong. A 93 per cent match to those labels shows the classifier imitates the old routing, which is why the business metric did not move; labels should come from the team that finally resolved each ticket.

Why D is wrong: Tempting because missing task context is a frequent cause of weak classification. It is wrong because the stem says the prompt already describes each team's remit, so the cause lies in how success was measured rather than in what the model was told.

Free sampleSolution Design & Architecturemedium

A hospital's revenue cycle team spends about 50 minutes on each prior authorisation request: pulling diagnosis and procedure codes from the record, matching them to the payer's published criteria, drafting a justification letter and submitting it once a clinician has signed. The steps are the same for every request, and hospital policy requires a clinician's signature before anything is sent to a payer. Engineering proposes an autonomous agent with record access and payer-portal submission rights. What should the architect recommend instead?

  • AThe autonomous agent as proposed, with every record access and portal submission written to an audit log for weekly compliance review
  • BThe autonomous agent with a clinician confirmation on every step it takes, including code lookups and each paragraph of the letter
  • CThe autonomous agent with submission rights removed, keeping open-ended planning so it can decide which steps each request requires
  • DA fixed workflow of defined steps in which Claude drafts the justification letter and a clinician signs before the system submits it Correct
Prefer a fixed workflow over an autonomous agent when steps are known in advance, placing human approval at the irreversible step. Autonomy pays off when the path through a task cannot be predicted. Prior authorisation here follows the same sequence every time, so a defined workflow with Claude drafting the letter gives predictable behaviour and simpler evaluation. Putting the clinician signature as a gate before submission meets the policy structurally rather than relying on detection afterwards.

Why A is wrong: It is tempting because audit logging is a familiar compliance control. It is wrong because logging only detects a submission made without a signature after the fact, whereas the policy requires the signature to come first, and the agent's autonomy is unnecessary for fixed steps.

Why B is wrong: It is tempting because it keeps a clinician involved throughout. It is wrong because it spends scarce clinician time on low-risk steps, erodes the time saving that justified the project, and still uses an agent where the steps are fixed.

Why C is wrong: It is tempting because removing the irreversible capability is sound practice. It is wrong because the stem states the steps are the same for every request, so open-ended planning adds cost and unpredictability without benefit when a fixed workflow fits.

Why D is correct: The steps are known in advance, so a fixed workflow is more predictable, cheaper and easier to evaluate than an open-ended agent, and placing the clinician signature before submission enforces the policy at the one irreversible step.

Free sampleSolution Design & Architecturemedium

A state government contact centre is piloting a Claude assistant for licence and registration enquiries. The executive sponsor will approve statewide rollout only if the pilot lowers the cost of resolving an enquiry, and defines an enquiry as resolved when the citizen does not contact the centre again about it within 14 days. The pilot team's draft report leads with containment rate, the share of conversations that end without transfer to a human agent. Which success measure should the architect put at the head of the report?

  • ATotal cost per resolved enquiry, counting model usage, human handling and any repeat contacts Correct
  • BContainment rate, because each conversation kept from a human agent removes that agent's handling cost
  • CAverage tokens consumed per conversation, because model usage is the cost the pilot directly adds
  • DAnswer accuracy on a reviewed sample, because a correct answer is what prevents a costly complaint
Lead a pilot report with the outcome metric that matches the sponsor's stated value pillar and definition, not a convenient proxy such as containment. A business value pillar is only demonstrated by a metric that measures it end to end. The sponsor chose cost and defined resolution by the absence of a repeat contact, so the measure must sum every cost incurred per resolution, including human handling and call-backs. Proxies such as containment, token spend or sample accuracy can each improve while the real cost per resolution worsens.

Why A is correct: This measures the pillar the sponsor named, cost, against the sponsor's own definition of resolution. It captures model spend, the human time still needed on transfers, and the cost of citizens coming back, so it moves only when the assistant genuinely lowers the cost of a resolution.

Why B is wrong: Containment is tempting because it is easy to measure and usually moves with cost. It is wrong here because a contained conversation that leaves the citizen to call back within 14 days is not resolved under the sponsor's definition, so containment can rise while cost per resolution rises too.

Why C is wrong: Token spend is tempting because it is the new line on the bill the pilot introduces. It is wrong because it leaves out human handling and repeat contacts, which are the costs the sponsor wants lowered and which a cheap but unhelpful assistant can increase.

Why D is wrong: Accuracy is tempting because a wrong answer does drive repeat contacts and complaints. It is wrong as the headline because it is a quality input rather than the cost outcome the rollout decision depends on; a high sample accuracy does not show that cost per resolved enquiry fell.

Other domains in this exam

See also the CCAR-P cert hub, the study guide, and the cheat sheet.

Examworthy is not affiliated with or endorsed by Anthropic. Original, blueprint-aligned practice material only.