A support assistant answers a routine question correctly in a demonstration. Before it reaches customers, the team needs to know how it handles missing information, conflicting guidance, and requests it should escalate. A release decision needs evidence from those cases too.
Turn requirements into testable outcomes
Suppose the assistant can explain published service information but cannot change an account or promise a refund. Its evaluation should test those boundaries alongside answer accuracy.
Write down expected behaviour before scoring outputs. Concrete criteria let reviewers distinguish a stylistic preference from a failure that affects the customer.
| Case | Expected behaviour | Evidence to inspect |
|---|---|---|
| Published information answers the question | Give an accurate answer supported by the approved source | Answer and source passage |
| Required information is missing | Ask for clarification or route the case for review | Missing information identified |
| Two approved sources conflict | Flag the conflict and escalate | Conflicting passages and routing result |
| Customer requests a refund commitment | Refer the decision to the authorised process | No invented commitment or unauthorised action |
Set the release consequences of different failures. An unsupported financial commitment may block release even when most answers are correct. Report these failures separately so an average score cannot hide them.
Build cases from the work
Use appropriately handled examples from the actual workflow, labelled by people who understand it. Include common enquiries, awkward wording, incomplete records, and known failures.
Add constructed cases where real examples are scarce, such as instructions hidden in a retrieved document or an unusual combination of policy conditions. Label these separately from observed traffic. A deliberately difficult test set helps find weaknesses but does not estimate how often those weaknesses occur in production.
Keep a separate holdout set for release assessment. Repeatedly editing prompts against the same examples can improve the test score without improving performance on unfamiliar cases. Sample size and coverage should reflect the decision's risk and the uncertainty the team can accept.
Use the right checking method
Use deterministic checks where the requirement is exact: valid output fields, allowed tool names, required identifiers, or whether a prohibited action occurred. Use human review where correctness depends on source interpretation or the appropriateness of an escalation.
A model judge can help review larger volumes, provided its scores are checked against human decisions. Give it a specific rubric with examples. Measure disagreement by failure type and examine whether it rewards confident wording or misses unsupported claims.
Check agreement between human reviewers too. A disputed label may reveal an unclear requirement. Keep human sampling after introduction of a judge, and reassess the judge when its model, rubric, or input distribution changes.
Evaluate the complete release
Record the model, prompt, retrieval sources, tools, and settings used in each run. A change to any of these can alter behaviour, so the release evaluation must cover the assembled system.
Compare the candidate with the current version. Inspect improvements, regressions, and unresolved failures. If a prompt change improves routine answers but weakens refund escalation, the aggregate gain does not settle the decision.
The release record should state the supported scope, criteria tested, results by failure type, known gaps, and the person accepting the result. Where evidence is insufficient, restrict the scope or collect more evidence.
Check that live behaviour matches the test
After release, monitor corrections, escalations, sampled answer quality, and changes in incoming work. Assign someone to investigate deviations and specify when to pause or roll back.
Confirmed incidents should inform regression coverage without exposing customer information unnecessarily. Keep the holdout evaluation distinct from cases used to tune fixes.
The practical output is a release decision supported by inspectable results. The team should be able to explain what passed, what remains uncertain, and how it will detect a deterioration.