CCAO-F domain - 21% of the exam

Output Evaluation and Validation

Output Evaluation and Validation is 21% of the CCAO-F exam. These are the objectives it covers, each with practice questions, with every answer explained.

The domain in numbers

56
Practice questions
6
Objectives
10
Easy
46
Medium

Where people go wrong

  • Treating Claude's own confirmation as verification

    The wrong answers ask Claude to vouch for itself: quote a confidence rating, run the sum again, or request a second pass. None of these is independent evidence, because a repeated pass can repeat the same misreading and a stated confidence does not locate errors. The correct answer goes back to the authoritative source, such as the original tender documents or the invoices, and compares each figure directly.

    See it in a sample question
  • Accepting a citation, link or exact figure as proof

    The wrong answers accept the look of a source: a named publisher, a citation, a working link or a page number. Claude can produce a real organisation's name, a year, a precise figure, a quote or a page location for something that does not exist, and a working link only shows that a page exists, not that it supports the claim. The correct answer opens the original document, confirms the figure, year and wording match, and removes the claim if no source turns up.

    See it in a sample question
  • Checking what is present and missing what is left out

    The wrong answers test the lines on the page: sample some rows, tick each sentence, or judge whether a count looks plausible. An omission leaves nothing to check, so every sentence can be accurate while a high-risk finding or a booking is absent. Completeness is judged by working from the source towards the output: tick each agreed action off the notes, confirm each key finding appears, and reconcile the total against the original list.

    See it in a sample question
  • Treating a disclaimer or a non-expert read as the review

    The wrong answers offer a review that cannot detect the risk: a proofread by another marketer, a note saying the draft may contain errors, or a lay reading of a legal clause. Review only helps when the reviewer can recognise the kind of error the content risks. A compliance claim, a VAT position or a termination clause goes to the person with that expertise, such as legal counsel, a tax adviser or a data protection specialist, before anyone acts on it.

    See it in a sample question
  • Choosing a layout that mirrors the source

    The wrong answers keep the source's shape or add polish: a dense table, the policy's chapter order, a contents page, colour coding, or reports kept one after another. The right layout follows the reader's task. A decision needed by Friday opens the message; a procedure where each answer sets the next question becomes ordered steps; a quick reference is headed by the situations clerks meet; a comparison groups views by issue.

    See it in a sample question

Objectives in this domain

What to study

The official documents this domain's practice questions cite most.

Sample questions from this domain

Free sampleOutput Evaluation and Validationmedium

A finance assistant at a logistics company uploads a spreadsheet of 40 fuel invoices and asks Claude for a one-page summary for the budget holder. The summary states that total fuel spend for the quarter was £184,350. The budget holder will use that total to decide whether to approve an overspend request. What should the assistant do with the total before sending the summary?

  • ATotal the invoices in the spreadsheet independently and compare the result with Claude's figure. Correct
  • BAsk Claude to add the invoices up again and confirm the new total matches its first answer.
  • CKeep the total, because Claude worked directly from the uploaded spreadsheet of invoices.
  • DRound the total to the nearest thousand so that a small error is unlikely to mislead anyone.
Verify a figure Claude has calculated from source data by recalculating it independently, not by trusting the source upload or asking Claude to repeat it. Totals and other derived figures can be wrong even when Claude has the source file, because a row can be skipped, misread or counted twice. Recalculating from the data is independent evidence; asking Claude to repeat the sum is not. When a decision depends on one number, that number earns the check.

Why A is correct: The decision turns on this one number, and the spreadsheet can produce it directly. An independent total from the source data confirms or corrects Claude's figure before the budget holder relies on it.

Why B is wrong: Repeating the sum seems like a second check. It is the same process checking itself, so a misread or skipped invoice can be repeated and the matching totals give false reassurance.

Why C is wrong: Working from the real file sounds safer than working from memory. Having the source does not guarantee every row was read and added correctly, so a derived figure still needs checking.

Why D is wrong: Rounding feels like a way to absorb small slips. It does not detect a large error such as a missed or doubled invoice, and it hides the figure the budget holder needs to judge the overspend.

Free sampleOutput Evaluation and Validationmedium

A marketing executive at a software firm is editing a Claude-drafted blog post for the company website. The draft says that a 2023 industry survey found 68 percent of small firms plan to increase their software spending. The firm's content policy requires every published statistic to be traceable to its original source. What should she do with this claim? Select TWO.

  • AAsk Claude to name the organisation that ran the survey and add that name as the source line
  • BLocate the original survey report and confirm that the figure, year and wording match it Correct
  • CAsk Claude how certain it is about the figure and keep the sentence if it says it is very sure
  • DSoften the claim to say about two-thirds of small firms, so that a small error does no harm
  • ERemove the statistic, or replace it with a sourced one, if the original report cannot be found Correct
A statistic in published copy must be checked against its original source and removed or replaced when that source cannot be found. Claude can generate realistic statistics and attributions that have no real source behind them. Finding the original report is the only way to confirm the number, and a claim that cannot be traced has to come out or be replaced, because no amount of rewording or self-assessment turns an unverified figure into a sourced one.

Why A is wrong: A named source looks like it meets the policy, but Claude may supply a plausible publisher that never ran such a survey, so the claim is still untraced.

Why B is correct: The original report is the authoritative source, and checking the figure, year and wording is what makes the statistic traceable as the policy requires.

Why C is wrong: A confident answer is reassuring, but self-rated certainty is not evidence and does not make the statistic traceable to anything.

Why D is wrong: Rounding feels like a safe compromise, but it still presents an unverified statistic and does nothing about a survey that may not exist.

Why E is correct: If the source cannot be found, the claim cannot meet the policy, so cutting it or substituting a verified figure is the correct outcome.

Free sampleOutput Evaluation and Validationmedium

A training coordinator at a regional charity pasted a list of 64 volunteer course bookings into Claude and asked it to reorganise them into a table grouped by course date. The table looks tidy, and the ten rows she spot-checked all match the original list. Volunteers missing from the table will not be sent joining instructions. Which check gives the best evidence that the table is complete?

  • ASpot-check another ten rows against the original list, so that a larger share of the table has been checked.
  • BAsk Claude to confirm that it has included each booking from the original list in the new table.
  • CCheck that each course date in the table is formatted consistently and sorted in the right order.
  • DCount the bookings in the table, check no name appears twice, and confirm the total matches the original 64. Correct
When Claude reformats or reorganises data, reconcile counts with the original, because sampling rows cannot reveal what was dropped. Reorganising a list should be lossless, yet entries can be dropped or duplicated without any visible sign in a tidy output. Sampling rows checks accuracy of what is present, not whether anything is missing. Reconciling the total against the original count, and checking for duplicates, gives direct evidence of completeness.

Why A is wrong: More sampling feels thorough, but it only confirms that rows present are correct. A booking that was dropped has no row to sample, so this cannot show the table is complete.

Why B is wrong: Asking Claude is quick, but its own confirmation is not independent evidence. It could report the table as complete while a booking is still missing.

Why C is wrong: Consistent dates make the table look finished, but presentation says nothing about whether any bookings were lost when the list was reorganised.

Why D is correct: A reorganisation should lose nothing, so reconciling the total with the original 64, and ruling out duplicates that could hide a gap, directly tests whether every booking made it across.

Free sampleOutput Evaluation and Validationmedium

A marketing executive at a software firm used Claude to draft a product page stating that the firm's platform 'meets every GDPR requirement for storing customer data in the UK'. A marketing colleague proofread the page and found no errors, so the executive now considers it reviewed and ready to publish on the company website. Which statement best explains why that proofread is not the review this page needs?

  • AContent drafted by Claude has to be reviewed by two people, so a second marketing colleague should also read the page.
  • BA proofread checks wording and clarity, but the risk here is a compliance claim, which only someone with data protection expertise can judge. Correct
  • CThe colleague should have asked Claude to confirm the GDPR claim, since that check would catch any legal inaccuracy in the draft.
  • DWebsite pages carry the firm's name, so the page needs sign-off from a senior manager rather than from a peer at the same level.
Review only counts when the reviewer has the expertise to detect the specific kind of error the content risks, not merely a second pair of eyes. Different content carries different risks, and each risk needs a reviewer who can recognise it. A proofread catches errors of expression, while a compliance claim on a public page can be fluent and still false. Matching the reviewer's expertise to the stakes of the claim is what makes the review meaningful.

Why A is wrong: It is tempting to treat review as a matter of how many people have looked at the page. Another marketing reader shares the first one's blind spot: neither can tell whether the GDPR claim is true, so adding readers adds no protection against the actual risk.

Why B is correct: The danger in this page is a public statement about legal compliance, and a proofreader is looking for spelling, grammar and flow. Review only helps when the reviewer can detect the kind of error the content risks, so the claim needs someone who understands data protection before it is published.

Why C is wrong: Asking Claude to confirm its own statement feels like a quick extra check. It is not independent evidence: the same model that produced the claim can confirm it just as confidently, and it cannot know how the firm actually stores customer data.

Why D is wrong: Seniority can seem like the missing ingredient for public content. A senior manager without data protection knowledge is no better placed than the proofreader to judge whether the compliance claim is accurate; what matters is the reviewer's expertise, not their rank.

Free sampleOutput Evaluation and Validationmedium

A project manager at a consultancy asked Claude to draft a status update for a client's executive sponsor. Claude produced a dense table covering twelve workstreams with a status for each. The sponsor reads updates on her phone between meetings and must approve a two-week extension to the project by Friday. What should the project manager ask Claude to change?

  • ARewrite it as short prose that opens with the decision needed, then gives the reason for it Correct
  • BAdd columns showing each workstream's full history so the sponsor sees how the delay built up
  • CTurn the table into a formatted artifact report with a cover page and a contents list added
  • DKeep the table but add the extension request as a closing note beneath the final row
Match the structure to the reader's task: when an executive must make one decision, lead with that decision in short prose rather than a dense table. A table suits comparison across many items, but this reader needs to act on a single request on a small screen. Putting the decision first, followed by the reason, organises the update around what the sponsor must do. Adding columns, reformatting into a longer report or appending the request at the end all leave the decision buried beneath detail she did not ask for.

Why A is correct: The sponsor's job is one decision by Friday, read on a phone. Leading with the request and a brief reason puts what she must act on first, in a form that reads easily on a small screen.

Why B is wrong: More detail can look like more transparency. It makes the table even harder to read on a phone and pushes the decision the sponsor must take further out of sight.

Why C is wrong: A polished report feels more professional for an executive. Adding structure and length does nothing to surface the extension request, and a multi-page report is awkward to read between meetings.

Why D is wrong: Keeping the existing work and adding the request seems efficient. Placing the one item that needs action at the bottom of a dense twelve-row table makes it the easiest part to miss.

Other domains in this exam

See also the CCAO-F cert hub, the study guide, and the cheat sheet.

Examworthy is not affiliated with or endorsed by Anthropic. Original, blueprint-aligned practice material only.