Trust plane / Phase 2
Governance, Oversight & AgentOps
What autonomy is allowed, and how does it evolve?
Decide how much independence each action may have, prove it with policy-as-code and human checkpoints, and run the change control that lets the system improve safely.
Use this harness in my journey →- Risk-based autonomy tiers: autonomy level as a function of action impact, uncertainty, identity, data sensitivity, regulation, reversibility and confidence
- Human-in-the-loop and human-on-the-loop checkpoints at durable pause points
- Policy-as-code compiled into gateway and runtime enforcement
- Registries for agents, prompts, models, tools, skills, datasets and policies
- CI/CD, canary and shadow release, with rollback
- Audit trail, lineage and evidence packs for regulators
- Feedback capture flowing into fine-tuning, distillation and prompt change
- Incident process and post-incident change
Accountable owner
AI Governance Office / Risk & Compliance, with Legal
- Chief Risk or Compliance Officer, or Head of Responsible AI — accountable owner
- AI risk manager — autonomy tiers and control design
- Compliance analyst — framework mapping and evidence
- Legal counsel — regulatory interpretation and contracts
- Product owner — accepts residual risk for their agent
- Internal audit — independent assurance
- Release manager — gates, canaries, rollback
Ownership and phase are architectural recommendations, not assessed results.
release gates consume evaluation evidence
incidents and cost signals
authors the rules guardrails enforce
approves write authority
redeploys versioned agents and policies
approval surfaces for reviewers
regulation-as-policy pairs with regulation-as-knowledge
agent and tool registries
Keep it lightweight in the pilot: an agent registry, an owner per agent and a written autonomy tier. Promote to full control the moment agents touch production systems or personal data — before any traffic, not after.
Signals to watch
- Share of actions executed inside the policy envelope
- Approval latency and override rate
- Registry completeness against deployed reality
- Control coverage mapped to the chosen framework
- Time to revoke or roll back an agent
- Share of incidents that produced a durable change
Governance owns promotion authority and the decision record. Changes to that authority require separate authorization.
How governed change crosses the architecture ↗Failure modes
- Approval theatre: reviewers who cannot see enough context to disagree
- Policy drift between the written standard and the deployed rules
- Shadow agents outside any registry
- Transparency obligations missed because the system was classified as low risk
- Feedback captured but never reaching a model or prompt change
Existing tool and standards snapshot · approximately mid-2026. Examples, not required dependencies; verify current versions before adoption.
Open source
- Open Policy Agent (Rego) or Cedar
- Argo CD and Argo Rollouts
- MLflow Model Registry
- Backstage as a service and agent catalogue
- Temporal for approvals as durable steps
- DataHub or OpenMetadata for audit lineage
Managed
- GRC platforms with AI control mappings
- Enterprise AI governance suites
- Model and agent registries from cloud vendors
Standards
- NIST AI Risk Management Framework plus the Generative AI Profile (Govern, Map, Measure, Manage)
- ISO/IEC 42001:2023 AI management systems
- EU AI Act — general application from 2 August 2026; Art. 50 transparency duties apply to any chatbot or generated content regardless of risk tier; Annex III high-risk obligations deferred to 2 December 2027 by the Digital Omnibus of 27 July 2026
- Sector regulation and internal model-risk policy
Autonomy is a policy, not a switch. The formula above lets one system be fully autonomous for reversible reads and strictly gated for irreversible writes, which is the only configuration that survives contact with a regulated environment.
