Skip to main content

Guide · Evaluation

Evaluate workflow agents against the job they must do

Test useful output, permissions, duplicate handling, failure and recovery before expanding an agent’s responsibility.

The short answer

Evaluate a workflow agent against a real operating task, not a convincing demonstration. Define the expected output and action boundary, build a varied test set, agree the evidence needed to accept it and inspect failures by consequence. Include changed inputs, duplicate events, denied access and interrupted runs. Increase responsibility only when the output remains useful and the operating team can detect, stop and recover from errors.

Define the task in testable terms

“Help the legal team with intake” cannot be evaluated. “Route each inbound contract request to the correct queue, summarize the request in three lines and list any missing information” can. A testable task names its inputs, the expected output, the actions the agent may take and the actions it must never take.

Write down what a good outcome looks like for an experienced person doing the same job today. That description becomes the reference the agent is measured against.

Build cases that challenge the design

Use approved historical records or synthetic cases with known answers. Include routine work, incomplete requests, conflicting instructions, changed priorities and cases that require escalation. Give reviewers the source record and the expected reasoning, not just a preferred answer sentence.

Create pairs that differ in one important fact. A request below an approval threshold should follow a different path from one above it. A revoked permission should change what can be retrieved. Two requests with the same wording but different customers must not share context. Hold back cases from tuning so the evaluation tests more than memorized examples.

Agree the rubric before you see results

Settle the scoring dimensions and what passes before the first run. Changing the rubric after seeing outputs is the easiest way to persuade yourself an agent is ready. Have at least two reviewers score a shared sample independently, and treat their disagreement as information about how clear the rubric is.

  • Correct routing or decision
  • Summary supported by the source material
  • Missing information flagged, not invented
  • No action outside the permitted scope
  • Escalation when the case calls for it

Test what happens after an action starts

Send the same event twice and confirm the workflow does not create two tickets or send two messages. Interrupt a run after the destination accepts an update but before the local receipt is saved. On retry, reconcile the destination state before executing again. A timeout does not prove that an action failed.

Also test denied access, a rate limit, expired authorization, a destination schema change and a reviewer who is unavailable. The useful result is a visible held item with the cause, current state, responsible owner and safe next step. A quiet retry loop or a green status with no verified destination output is a failed result.

Classify failures, not just count them

A pass rate on its own hides what matters. Sort every failure into a small set of types: used the wrong source, made an unsupported claim, missed an escalation, exceeded its permission, or formatted badly. Then weight them by consequence. An unsupported claim that could reach a client is not equivalent to a formatting slip.

The failure record is also the to-do list. Each type points to a different fix, whether that is source access, clearer instructions, a tighter permission or an extra review gate.

Stage autonomy with explicit exit criteria

Begin with observation: the agent prepares output while people complete the existing process. Compare usefulness, omissions and review effort. Next, allow it to prepare work for approval. Expand only the specific actions supported by the evaluation and the owner’s authorization; authority is a permission decision, not a model confidence score.

Name the person who can pause the workflow and the conditions that trigger a return to a narrower mode. Recheck affected cases after changes to the model, rules, tools, data or access. Preserve human approval before external messages, financial commitments or other consequential actions where the operating policy requires it.

Judge the output as a work product

Ask a person who did not prepare the test whether they can act on the result. A summary that repeats the request but misses the decision, owner or due date is not a useful intake record. A detailed answer that hides an unresolved contradiction is not ready for approval.

Track correctness and usefulness separately from response time and cost. Record what the reviewer had to look up, rewrite or ask another person. These observations reveal whether the workflow removes work or merely moves it.

Worked example

Sample artifactEvaluation scorecard
Task
Contract-request intake: route, summarize, list missing information
Test set
Approved historical requests · held-out portion untouched during tuning
Rubric
Five dimensions agreed before the run · two independent scorers
Results
Record task accuracy, source support, useful completeness and reviewer disagreement for each case
Failures
Classified by type and weighted by consequence
Stage decision
Stay in shadow, move to assisted, or step back, recorded with the owner's name
Sample evaluation scorecard. Use it to record an actual test round; no performance results are asserted here.

Checklist

  1. Task written with inputs, outputs, allowed and forbidden actions
  2. Reference description of a good human outcome
  3. Test set from approved past cases, including hard ones
  4. Held-out cases kept apart from tuning
  5. Rubric and pass criteria fixed before the first run
  6. Two independent scorers on a shared sample
  7. Failures classified and weighted by consequence
  8. Stage entry criteria, rollback triggers and re-evaluation rules documented
  9. Duplicate and interrupted runs reconcile destination state before repeating an action
  10. Failure produces a visible owner, current state and next step
  11. An independent reviewer can use the output without rebuilding it

Review responsibilities

Process owner
Defines the task and the good outcome, and makes the stage decision.
Evaluators
Score outputs independently against the rubric and record their reasoning.
Risk and compliance
Sets the actions that always need human approval and reviews severe failures.
Platform owner
Enforces permissions, logs every run and triggers re-evaluation after changes.

Limitations

  • A test set cannot cover every future request. Monitor new failure types and add representative cases.
  • A second model can assist review but can share the first model’s errors; source checks and human judgment still matter.
  • A high aggregate pass rate can hide a serious permission or action failure. Review critical categories separately.
  • Changing a model, rule, source or integration can change behavior. Retest the affected workflow rather than transferring an old score.

FAQ

How many test cases do we need?

Enough to cover the routine, the awkward and the escalated cases, with a held-out portion. The right number depends on how varied the workflow is, and it is better agreed in discovery than fixed in advance.

Can the agent grade its own work?

Automated checks can help with formatting and whether a source link exists. Judging whether a summary is correct or an escalation was right should stay with human reviewers.

When should we re-evaluate?

After any change to the model, instructions, sources or permissions, and on a regular schedule even when nothing appears to have changed.

What if the agent outscores people on the test set?

Treat it as a reason to check the test set first. Even a strong result moves an agent one stage at a time and never removes human approval for consequential actions.

Put the guidance to work

Bring the business goal, systems, source owners and operating constraints. We use the conversation to identify a useful scope and the next architecture or implementation step.

Discuss your architecture

Describe the system or process you want to connect. HAI reviews each request and agrees the scope with you.

Next guideData foundations for useful AI