How to Write an AI Evaluation Rubric Your Business Owner Can Approve

The short answer#

Write an AI evaluation rubric around one business task, with observable criteria and examples for each score. Separate critical failures from quality grades, then ask business reviewers to score the same cases independently. Resolve disagreements and agree the release rule before testing the candidate, with a named owner for future changes.

Start with a decision your business owner can defend#

If your delivery team calls an answer accurate while the operations owner calls it unusable, a green dashboard will not settle the release decision. The disagreement will return during acceptance, consuming review time and delaying approval.

A rubric is a written guide to grading the result. It tells reviewers what evidence to inspect, what each score means and which failures prevent acceptance. A label such as “helpful, 1 to 5” leaves too much for each reviewer to invent. Anthropic's January 2026 evaluation guide warns that ambiguous criteria can make grading inconsistent.

Start with one sentence describing the work and its boundary. Consider an internal purchase-policy assistant: it explains the approved purchasing process to employees, using documents they may access. It cannot approve a purchase or place an order.

The business outcome is that an employee can prepare the right approval request without asking procurement to repair the instructions. That gives the business owner something concrete to assess: the answer's accuracy, required information and next step.

This worked example uses a fictional policy. Its responses, scores, workload estimates and release targets are planning examples, not client results or industry benchmarks.

Give the reviewer evidence before asking for a score#

An unsupported grade cannot explain why a release passed. For every test case, give the reviewer a small evidence pack:

• The employee's question and access role.
• The authorised policy version and relevant passages, including any applicable exceptions.
• The expected decision: answer, ask for missing information or refer to a named policy owner.
• A checklist of the facts and steps the response must contain, plus unacceptable statements.
• The assistant's actual response and, where relevant, records of documents accessed or actions attempted.

For the worked case, suppose the approved fictional policy says: a new software purchase requires manager and procurement approval before ordering; the employee submits a supplier quotation with the request in the purchasing portal. The employee asks, “What do I need to do before ordering new software?”

The expected response must include both approvals, the timing, the quotation and the submission route. Write that checklist before looking at the assistant's answer. If policy documents conflict, have their owner resolve the conflict or specify the correct referral; reviewers should not invent the policy while grading.

Use the same discipline for missing information. If the applicable process depends on a detail the employee has not supplied, the expected response should identify that detail and ask for it. A confident answer to an unresolved question should lose credit.

Put critical failures outside the quality total#

A clear explanation does not compensate for exposing restricted purchasing information. Define the events that stop release separately from the quality score.

For this assistant, propose two critical failures:

• It reveals information the employee is not permitted to access.
• It states that a purchase is approved, or attempts an approval or order, despite having no authority to do so.

An answer that says “Your purchase is approved; order it today” triggers the second failure. Record it as a blocker even if the rest of the response is well written. Continue recording quality scores if they help diagnose the problem, but never let their total override the blocker.

The delivery team must also test access enforcement and prohibited actions in the application. A reassuring answer alone cannot establish that restricted documents were never retrieved or an order was never attempted.

Use a small rubric with observable scoring anchors#

The business owner needs to recognise the difference between grades. Start with three dimensions and a 0–2 scale for each. Keep the dimensions separate in the report, even if you also show a total out of six.

These anchors apply to the software-purchase case. Add equally explicit anchors for other case types, including exceptions and referrals. For a missing-policy case, full credit might mean naming the uncertainty and directing it to the authorised policy owner. Do not reward invented certainty to preserve a high answer rate.

Now grade this response:

Before ordering, obtain approval from your manager and procurement.

It earns 2 for correctness, because its claims match the fictional policy; 1 for coverage, because the quotation is missing; and 1 for the next step, because it does not explain submission. The total is 4/6. That makes the repair visible: add the quotation and portal instructions.

A complete response could say:

Submit a request in the purchasing portal with the supplier quotation. Obtain manager and procurement approval before placing the order.

That earns 2/2/2 under these anchors. Equivalent wording should receive the same grades: the reviewer checks meaning against the evidence, rather than requiring a memorised sentence.

Check whether two reviewers mean the same thing by “pass”#

Inconsistent reviewers can make the system appear to improve when only the grading has changed. Google Cloud's evaluation guidance notes that human evaluation is also affected by the task and criteria.

Before accepting release results, run a calibration meeting. As a practical starting proposal, have two business reviewers independently score ten shared examples covering a complete answer, a plausible omission, an incorrect answer, a justified clarification and a critical failure. Keep their initial scores hidden from each other.

Compare each criterion and the blocker decisions. For a disagreement, ask each reviewer to point to the answer, the policy passage and the scoring anchor. Record whether the cause was a vague anchor, missing evidence, a policy ambiguity or a reviewer mistake. Revise the relevant example or instruction, then independently grade a fresh shared batch.

Report agreement as a count with a denominator: for example, matching scores on 24 of 30 criterion decisions, plus a separate count for critical-failure decisions. A small exercise diagnoses ambiguity; it does not establish statistical reliability.

Name one business owner to settle unresolved interpretations. Do not average two disputed scores and declare the issue resolved. Save the rubric version, original grades, final decisions and reasons so later results remain interpretable.

If you later introduce an AI grader to reduce review effort, compare it against examples whose grades the business owner has settled. Anthropic's guide recommends checking model-based grading against human expertise. Keep people responsible for disputed or high-consequence decisions.

Agree the release rule before seeing the result#

A score becomes useful when everyone knows what decision follows it. For this fictional assistant, you could propose the following rule for a limited pilot with human review:

• No critical failures in any executed test, including repeated trials of important cases.
• Every accepted routine response scores 2 for correctness and at least 5/6 overall.
• At least 90 of 100 reserved routine cases, kept back and never used for tuning, produce an accepted response; report the actual counts and each dimension's results.
• Every case in a separately listed critical challenge set, covering deliberately difficult and prohibited requests, produces its specified permitted behaviour. List every failure individually.

These are illustrative targets for discussion. The business and risk owners must decide whether the consequences of a wrong answer justify stricter criteria, narrower scope or further evidence. This rule covers answer acceptance for the pilot; access controls, operating support and the other release requirements still need their own approval.

Select routine cases from the work employees will actually bring. Google Cloud's dataset guidance recommends reflecting production inputs. Include difficult routine variations, and keep cases used for tuning separate from the reserved acceptance set.

Report challenge cases separately so their deliberately high concentration of exceptions does not distort the routine acceptance rate. Track unnecessary referrals as well: an assistant that sends every answerable question to procurement preserves the workload it was meant to reduce.

Passing the selected tests provides bounded evidence. It cannot prove that every future response will be safe or correct. Keep sampling pilot work, record new failures and decide who can restrict use when a critical problem appears.

What this means for you#

Bring the business owner, delivery lead and risk owner together with a draft rubric and a small set of real, appropriately protected cases. Leave the meeting with an agreed task boundary, scoring examples, a disagreement log, a release rule and named owners. If the policy itself is unsettled, resolve that before asking engineering to optimise the assistant.

The business owner owns the expected decisions and quality interpretation. The delivery lead owns repeatable execution, evidence capture and reporting. The risk owner owns the critical-failure definitions; keep their clearance explicit in the release decision.

Budget business-review time as part of delivery. For the ten shared cases, reserve a two-hour calibration meeting. If grading takes three minutes per case, each reviewer needs five hours to grade 100 reserved cases independently: ten hours across two reviewers, plus the meeting and time to resolve disagreements. Time the first ten cases and replace that planning assumption with the observed effort before committing the budget.

An unclear rubric can force both reviewers to grade the same 100 cases again after the criteria change, repeating that ten-hour grading allocation before acceptance can finish. Automate checks that have objective answers, and reserve business judgement for the decisions that require it.

Keep the rubric, cases and grading history under your organisation's control when buying a tool or commissioning a consulting engagement. Ask for a handover that lets your team update them when policy changes. Version every change and regrade affected cases; do not compare scores from different rubric versions as though the standard were unchanged.

Once acceptance is defined, the Article on evaluation before a model upgrade explains how to use the evidence to decide which part of the application to improve.

Questions to ask before signing off the rubric#

• Can two business reviewers explain the same grade using the same evidence?
• Which failures block release regardless of the total score?
• What does a correct referral look like, and how will we spot unnecessary referrals?
• Were the cases and thresholds agreed before the candidate's results were known?
• Who resolves disagreements, maintains the rubric and acts on failures during the pilot?

If your team needs help turning business acceptance criteria into a repeatable evaluation and release decision, Discuss your project.