Evaluation & Assurance

• Five evaluation scopes: component, agent, multi-agent trajectory, system, and trust
• Trace-based grading — LLM-as-judge with calibration, agent-as-a-judge over the trajectory, human grading
• N-run reliability: median, variance and cost-adjusted completion across repeated runs
• Regression suites built from production incidents
• Adversarial and red-team testing, including injection and harmful-action benchmarks
• Shadow and canary evaluation before promotion
• Simulation of users and environments for rehearsal at scale

Quality / AI Assurance, with the Security red team

• Head of Quality or AI Assurance — accountable owner
• Evaluation engineer — harnesses, datasets, judges
• QA lead — regression discipline and release criteria
• Red teamer — adversarial campaigns
• Domain expert grader — the ground truth for quality
• Data scientist — judge calibration and statistics

Architectural recommendation, not a research finding.

Pilot needs a golden set and a regression suite fed by real incidents. Adversarial campaigns arrive with production; simulated users and environments are the phase 3 workstream for rehearsing at scale.

Signals to watch

• Pass rate by evaluation scope
• Variance across N runs, not just the median
• Cost-adjusted completion rate
• Attack success rate with the harness and judge configuration attached
• Judge-to-human agreement (kappa) on a sampled set
• Share of production incidents converted into regression cases

1. Single-run scores treated as evidence: a good demo says almost nothing about production
2. Evaluation awareness — a system that behaves differently when it senses it is being tested, inflating measured safety
3. Uncalibrated LLM judges disagreeing with humans in exactly the cases that matter
4. Benchmark contamination and overfitting to public sets
5. No domain expert in the loop, so plausible-but-wrong passes

Tool and standards snapshot · last reviewed approximately mid-2026