of deployed enterprise agents actively monitored and secured in Gravitee’s 2026 survey.
Primary report ↗
Assess readinessEnterprise multi-agent systems
The model was never the hard part.
Sixteen harnesses turn a capable model into an enterprise system that can be operated, governed, and trusted.
04 concern planes16 harnesses43 exchanges01 governable system
RESEARCH SNAPSHOT / REVIEWED 01 SEP 2026
of organisations unable to clearly separate agent activity from human activity.
CSA research ↗the tokens of a chat interaction for one observed production multi-agent research system.
Anthropic engineering ↗exchanges in this reference task that touch the model itself; the rest belong to the harness.
Trace the task ↗01 / THE IDEA IN ONE PICTURE
The model is the engine.
The harness is the vehicle.
Intelligence is only one component. Around it sit the runtime, knowledge, execution, and trust concerns that let an enterprise put real work—and real consequences—through the system.
The rings group concerns. They do not imply priority, containment, or dependency.
Runtime plane
1ComputeWhere does it run?
The compute, scheduling, networking and resilience substrate every other harness sits on — Kubernetes across public and private cloud, GPU-aware and multi-tenant from the start.
Ingredients
- Kubernetes clusters across public and private cloud, with a private path for regulated workloads
- GPU scheduling, quota and gang-scheduling so accelerators are not idle
- Storage, queues, service mesh, secrets distribution and network policy
- High availability, disaster recovery and capacity planning
- Tenancy primitives: namespaces, node pools, network segmentation
2ModelWhich intelligence executes?
Serve, version and scale the models — proprietary and open-weight — with predictable latency and cost, and keep them swappable so no single provider becomes load-bearing.
Ingredients
- Inference serving: batching, KV cache, tensor and pipeline parallelism
- Model registry and version pinning, including deprecation calendars
- Routing between model tiers, plus fallbacks when a provider degrades
- Embedding and reranking endpoints used by retrieval
- Capacity and quota per tenant on shared accelerators
3GatewayHow is model traffic controlled?
One control point for every model call: authentication, tenant quota, routing, caching, cost metering and the edge layer of guardrail enforcement.
Ingredients
- Authentication and authorisation for callers, human and agent
- Per-tenant quotas, rate limits and budget ceilings
- Semantic and cost-aware routing across model tiers and providers
- Response and prefix caching
- Token metering and chargeback per team, tenant and task
- Edge guardrail hooks: request and response screening before anything reaches a model
4InteractionHow does a human participate?
The surfaces where people meet the system — and where they steer it: channels, streaming, sessions, clarification, provenance display and the approval moments that make autonomy safe.
Ingredients
- Channels: web, mobile, chat platforms, voice and public API
- Streaming and partial-result rendering so long tasks stay legible
- Session and conversation state at the edge
- Clarification prompts when the task is under-specified
- Approval and review surfaces for human-in-the-loop checkpoints
- Provenance display: what the answer was based on and how confident it is
- Multimodal input and output, and accessibility
Trust plane
5Secure&SafeWho or what is allowed to do it?
Runtime enforcement on every call, plus the agentic attack surface: non-human identity, injection and PII defence, output validation, supply chain, circuit breakers and kill switches.
Ingredients
- Deterministic checks pre-call, in-loop and post-call
- Prompt-injection and jailbreak defence, including on retrieved content
- PII and PHI detection and redaction in both directions
- Output validation against schema and policy before anything executes
- Non-human identity: one identity per agent, tool and server; just-in-time scoping
- Supply-chain controls: signing, scanning, pinning, SBOM and AIBOM
- Circuit breakers and kill switches with a tested revocation path
6GovernanceWhat autonomy is allowed, and how does it evolve?
Decide how much independence each action may have, prove it with policy-as-code and human checkpoints, and run the change control that lets the system improve safely.
Ingredients
- Risk-based autonomy tiers: autonomy level as a function of action impact, uncertainty, identity, data sensitivity, regulation, reversibility and confidence
- Human-in-the-loop and human-on-the-loop checkpoints at durable pause points
- Policy-as-code compiled into gateway and runtime enforcement
- Registries for agents, prompts, models, tools, skills, datasets and policies
- CI/CD, canary and shadow release, with rollback
- Audit trail, lineage and evidence packs for regulators
- Feedback capture flowing into fine-tuning, distillation and prompt change
- Incident process and post-incident change
7ObservabilityWhat happened — and what did it cost?
Make any task reconstructable end to end and attribute its cost: OpenTelemetry GenAI spans carrying trace, thread and task IDs, the acting identity, the policy decision and the token spend.
Ingredients
- Distributed tracing across model calls, tool executions, retrieval, memory and agent runs
- Correlation IDs that survive handoffs, queues and human approvals
- Time-travel debugging: replay of prompts, delegations and memory state at failure
- Quality and drift monitoring, including silent-failure detection
- Token metering, budgets, chargeback and cost per successful task
- Agent-versus-human attribution in every log
8EvaluationWas it good, safe and compliant?
Grade the system at five scopes, prove reliability across repeated runs, red-team it adversarially, and gate every promotion on the evidence.
Ingredients
- Five evaluation scopes: component, agent, multi-agent trajectory, system, and trust
- Trace-based grading — LLM-as-judge with calibration, agent-as-a-judge over the trajectory, human grading
- N-run reliability: median, variance and cost-adjusted completion across repeated runs
- Regression suites built from production incidents
- Adversarial and red-team testing, including injection and harmful-action benchmarks
- Shadow and canary evaluation before promotion
- Simulation of users and environments for rehearsal at scale
Execution plane
9ProtocolHow do agents, tools and peers talk?
Standard contracts at every boundary so frameworks stay replaceable: MCP downward to tools and data, A2A sideways to peer agents, OpenAPI and AsyncAPI to the rest of the estate.
Ingredients
- MCP servers and clients for tools, resources and prompts
- A2A agent cards, task lifecycle and artifacts for agent-to-agent work
- OpenAPI and AsyncAPI contracts for conventional services and events
- Registries: which agents, tools and skills exist, at which version, owned by whom
- Version and deprecation policy across protocol revisions
- Supply-chain scanning of third-party servers and skills
10OrchestrationWho acts, when and in what order?
Plan and decompose the task, route work to the right agent or tool, checkpoint durably so long tasks survive restarts, and stop cleanly when budget or policy says so.
Ingredients
- Planning and decomposition; orchestrator-worker and handoff patterns
- Durable checkpoints, retries and idempotency for side-effecting steps
- Termination policy: step limits, budget ceilings, loop detection
- State machines that make agent behaviour inspectable
- Multi-agent topology decisions, including when not to use one
- Resumption after human approval without replaying side effects
11ActionWhat can the system actually do?
The action surface: typed tool contracts, packaged skills, an explicit read-versus-write authority model, and isolation strong enough for code the model wrote.
Ingredients
- Tool contracts with typed inputs, outputs and error semantics
- Skills as packaged, versioned capability (SKILL.md and equivalents)
- Read-versus-write authority classification for every tool
- Sandboxing for generated code and untrusted execution
- Idempotency, dry-run and rollback for side-effecting tools
- Tool catalogue, ownership and deprecation
12MLWhen is a classical model the better tool?
Keep discriminative and optimisation models in the architecture as first-class tools and routing signals — often cheaper, faster and more accurate than asking a language model.
Ingredients
- Classifiers for routing, triage, intent and safety pre-screening
- Rerankers and scorers supporting retrieval
- Anomaly detection and forecasting feeding agent decisions
- Optimisation and constraint solving where the answer is a plan, not prose
- Feature store, training pipelines and drift monitoring for these models
Knowledge plane
13DomainWhat does the domain mean?
Shared, machine-usable meaning: ontologies, a semantic layer, agreed metric definitions and regulation expressed as knowledge the agent can reason over.
Ingredients
- Domain ontology and taxonomy — entities, relationships, business rules
- Semantic layer and metric definitions so one term returns one number
- Glossary and synonym mapping between business and system vocabulary
- Regulation as knowledge: rules the agent must understand, distinct from policy it must obey
- Stewardship: who owns each definition and how changes are reviewed
14DataWhere does enterprise truth come from?
Governed access to source systems with lineage and entitlements attached — the data plane behind retrieval, deliberately separate from the tool plane that takes actions.
Ingredients
- Connectors and data servers exposing systems of record to agents
- Data contracts: schema, freshness and quality guarantees per source
- Lineage from source through transformation to the answer
- Entitlements: row and column security carried through to the agent
- Federated query across warehouses, lakes and operational stores
- Classification of sensitive fields before anything is indexed
15RetrievalWhat information matters now?
Assemble the smallest sufficient context for each step, inside a token budget: hybrid retrieval, reranking, graph traversal, iterative refinement, compaction and tool search.
Ingredients
- Hybrid retrieval (lexical plus dense) with reranking
- GraphRAG and structured retrieval where relationships matter
- Agentic, iterative retrieval that refines its own queries
- Context pruning and compaction to fight bloat
- Token budgeting per step and per task
- Tool search: retrieving relevant tool definitions instead of loading all of them
16MemoryWhat should the system remember?
Decide deliberately what persists beyond a task — working, episodic, semantic and procedural memory — with an explicit write policy, retention limits and poisoning controls.
Ingredients
- Working memory within a task and session
- Episodic memory: what happened before, per user and per tenant
- Semantic memory: durable facts learned about the domain and the user
- Procedural memory: how this organisation does a task
- Write policy — what earns a place in memory — plus TTL and forgetting
- Redaction, tenant partitioning and the right to erasure
02 / TWO OF FORTY-THREE
Only two exchanges touch the model.
Everything else establishes identity, context, authority, evidence, safety, continuity, cost, and accountability.
01U → EXPRtask arrives on a channel02EXPR → GWsession and tenant context03GW → GUARDscreen before spending a token04GUARD → GWcleared · scoped agent identity05GW → ORCHplan, inside this budget06ORCH → DOMwhat do these terms mean?07DOM → ORCHresolved entities and metrics08ORCH → RETassemble context for this step09RET → DATAgoverned rows and columns10DATA → RETdata with lineage and entitlements11RET → MEMwhat do we already know?12MEM → RETepisodic, semantic, procedural13RET → ORCHcontext inside the token budget14ORCH → GWreason over this context15GW → INFERrouted, cached, metered16INFER → COREinference on a pinned model17CORE → INFERcompletion18INFER → ORCHnext action, schema-valid19ORCH → GOVscore against autonomy policy20GOV → EXPRapproval at a durable checkpoint21EXPR → GOVhuman decision22GOV → ORCHproceed unattended, logged23ORCH → TOOLexecute the typed tool call24TOOL → INFRAprovision an isolated sandbox25INFRA → TOOLmicroVM, gVisor or Wasm ready26TOOL → MLHclassify, rerank or forecast27MLH → TOOLscore or plan, far cheaper28TOOL → PROTOcross on a standard contract29PROTO → EXTMCP to systems · A2A to peers30EXT → PROTOresult or task handle31PROTO → TOOLtyped result or typed error32TOOL → ORCHobserve, then re-plan33ORCH → MEMwrite back what earns persistence34ORCH → GUARDvalidate the final output35GUARD → GWschema-valid and policy-clean36GW → EXPRmetered and charged to the tenant37EXPR → Uanswer, with provenance38ORCH → OBSa span for every hop39OBS → EVALtraces become eval cases40EVAL → GOVevidence for the release gate41GOV → ORCHredeploy versioned artefacts42GOV → GUARDpolicy compiled into enforcement43INFRA → OBScapacity, cost and availability
03 / THE BLUEPRINT AT A GLANCE
Four concerns. Sixteen accountable boundaries.
Runtime plane
Knowledge plane
Execution plane
Trust & lifecycle plane
04 / CHOOSE YOUR READING DEPTH
One architecture. Three ways in.
05 min
Executive
See why pilots stall, what the missing system costs, and what to build first.
Read the brief ↗30 minArchitect
Review all sixteen boundaries, integration points, ownership, and sequencing.
Open the blueprint ↗ReferenceEngineer
Inspect the detailed contracts, signals, standards, and end-to-end exchanges.
Use the explorer ↗