Evaluation & Assurance

• Five evaluation scopes: component, agent, multi-agent trajectory, system, and trust
• Trace-based grading — LLM-as-judge with calibration, agent-as-a-judge over the trajectory, human grading
• N-run reliability: median, variance and cost-adjusted completion across repeated runs
• Regression suites built from production incidents
• Adversarial and red-team testing, including injection and harmful-action benchmarks
• Shadow and canary evaluation before promotion
• Simulation of users and environments for rehearsal at scale

Accountable owner

Quality / AI Assurance, with the Security red team

• Head of Quality or AI Assurance — accountable owner
• Evaluation engineer — harnesses, datasets, judges
• QA lead — regression discipline and release criteria
• Red teamer — adversarial campaigns
• Domain expert grader — the ground truth for quality
• Data scientist — judge calibration and statistics

Ownership and phase are architectural recommendations, not assessed results.

Pilot needs a golden set and a regression suite fed by real incidents. Adversarial campaigns arrive with production; simulated users and environments are the phase 3 workstream for rehearsing at scale.

Signals to watch

• Pass rate by evaluation scope
• Variance across N runs, not just the median
• Cost-adjusted completion rate
• Attack success rate with the harness and judge configuration attached
• Judge-to-human agreement (kappa) on a sampled set
• Share of production incidents converted into regression cases

Evaluation supplies independent evidence. Protect held-out tests and calibrate evaluator changes separately from the candidate they judge.

Failure modes

1. Single-run scores treated as evidence: a good demo says almost nothing about production
2. Evaluation awareness — a system that behaves differently when it senses it is being tested, inflating measured safety
3. Uncalibrated LLM judges disagreeing with humans in exactly the cases that matter
4. Benchmark contamination and overfitting to public sets
5. No domain expert in the loop, so plausible-but-wrong passes

Existing tool and standards snapshot · approximately mid-2026. Examples, not required dependencies; verify current versions before adoption.

A system can have an excellent model and still be a poor agent. Only trajectory-level and repeated-run evaluation surfaces that difference — and it is the difference between a pilot and a product.