Trust & lifecycle plane / Tier 2 / Phase 2
Governance, Oversight & AgentOps
What autonomy is allowed, and how does it evolve?
Decide how much independence each action may have, prove it with policy-as-code and human checkpoints, and run the change control that lets the system improve safely.
- Risk-based autonomy tiers: autonomy level as a function of action impact, uncertainty, identity, data sensitivity, regulation, reversibility and confidence
- Human-in-the-loop and human-on-the-loop checkpoints at durable pause points
- Policy-as-code compiled into gateway and runtime enforcement
- Registries for agents, prompts, models, tools, skills, datasets and policies
- CI/CD, canary and shadow release, with rollback
- Audit trail, lineage and evidence packs for regulators
- Feedback capture flowing into fine-tuning, distillation and prompt change
- Incident process and post-incident change
release gates consume evaluation evidence
incidents and cost signals
authors the rules guardrails enforce
approves write authority
redeploys versioned agents and policies
approval surfaces for reviewers
regulation-as-policy pairs with regulation-as-knowledge
agent and tool registries
AI Governance Office / Risk & Compliance, with Legal
- Chief Risk or Compliance Officer, or Head of Responsible AI — accountable owner
- AI risk manager — autonomy tiers and control design
- Compliance analyst — framework mapping and evidence
- Legal counsel — regulatory interpretation and contracts
- Product owner — accepts residual risk for their agent
- Internal audit — independent assurance
- Release manager — gates, canaries, rollback
Architectural recommendation, not a research finding.
Keep it lightweight in the pilot: an agent registry, an owner per agent and a written autonomy tier. Promote to full control the moment agents touch production systems or personal data — before any traffic, not after.
Signals to watch
- Share of actions executed inside the policy envelope
- Approval latency and override rate
- Registry completeness against deployed reality
- Control coverage mapped to the chosen framework
- Time to revoke or roll back an agent
- Share of incidents that produced a durable change
- Approval theatre: reviewers who cannot see enough context to disagree
- Policy drift between the written standard and the deployed rules
- Shadow agents outside any registry
- Transparency obligations missed because the system was classified as low risk
- Feedback captured but never reaching a model or prompt change
Tool and standards snapshot · last reviewed approximately mid-2026
Open source
- Open Policy Agent (Rego) or Cedar
- Argo CD and Argo Rollouts
- MLflow Model Registry
- Backstage as a service and agent catalogue
- Temporal for approvals as durable steps
- DataHub or OpenMetadata for audit lineage
Managed
- GRC platforms with AI control mappings
- Enterprise AI governance suites
- Model and agent registries from cloud vendors
Standards
- NIST AI Risk Management Framework plus the Generative AI Profile (Govern, Map, Measure, Manage)
- ISO/IEC 42001:2023 AI management systems
- EU AI Act — general application from 2 August 2026; Art. 50 transparency duties apply to any chatbot or generated content regardless of risk tier; Annex III high-risk obligations deferred to 2 December 2027 by the Digital Omnibus of 27 July 2026
- Sector regulation and internal model-risk policy
Autonomy is a policy, not a switch. The formula above lets one system be fully autonomous for reversible reads and strictly gated for irreversible writes, which is the only configuration that survives contact with a regulated environment.