Most agent architectures fail in the same place: nobody decided where control lives.
The model picks tools. The prompt grants permissions. The harness keeps whatever state it happens to keep. Each choice is reasonable alone; together they produce a system where you cannot answer why did this run touch that file? without reading a transcript.
An orchestration layer is the answer to where control lives. This page covers what belongs in it, what does not, and what breaks when the boundary is unclear.
How the Layer Differs from Agents, Workflows, and Platforms
Four things get called "orchestration" and only one of them is the layer.
An agent does the work — reasons about a task, calls tools, produces output. It is the thing being controlled.
A workflow is one sequence for one job: this repository, this kind of change, these checks. Workflows are configuration expressed in the layer's terms; they are not the layer.
A platform is the commercial packaging: hosting, billing, accounts, SSO, a UI. A platform usually contains an orchestration layer, but conflating the two is how teams end up unable to change vendors.
The layer is the control plane. It decides which model runs a step, which tools that step may call, what state survives it, and who approves before it lands.
A practical test: if a property changes when you swap the model or the tool set — but the task itself is unchanged — that property belongs in the layer. Which model ran step three is a layer concern. How to fix the bug is not.
Core Control Responsibilities
Model Routing
Routing is the decision of which model handles which step, and it needs to be a decision rather than a default.
Criteria worth encoding: capability required for the step, cost per call, prompt size against the model's context and pricing thresholds, availability and fallback, and any data-residency constraint.
Two properties matter more than the routing logic itself. Observability: every step records which model handled it and the surrounding operation context — without that, an evaluation from last month cannot be interpreted this month, because you no longer know what it measured. Reversibility: routing changes revert without redeploying. From the inside, a silent routing change is indistinguishable from a model regression.
Tool and Permission Boundaries
The layer decides which tools exist, which are callable in which context, and against which resources.
The common anti-pattern is putting this in the prompt. Instructing a model not to touch production is a request, not a boundary — it competes with every other instruction in the window and loses to a sufficiently confident plan.
Boundaries belong where they are enforced independently of what the model decides. Three things the layer should own:
- Which tools are registered at all for a given task class, including tools exposed through a protocol such as MCP
- The default posture — whether an unlisted action is denied or allowed, and which specific actions require confirmation
- Resource scope — which repositories, credentials, and environments a run can reach, regardless of what the task appears to require
That last one is where the real damage happens. A correct agent with an over-broad credential is more dangerous than a confused agent with a narrow one.
Task State and Memory
"State" covers three different things with three different lifecycles, and merging them causes most of the confusion in this area.
Run state steps, retries, tool results, artifacts lives for one run
Task state goal, constraints, acceptance criteria lives until the task closes
Durable memory conventions, preferences, prior decisions lives across tasksThe layer owns the lifecycle and scope of each. Two questions decide whether the design is sound.
What survives a failed step? If a retry loses the task's acceptance criteria, the retry is a different task wearing the same name.
What crosses boundaries? Durable memory that silently spans repositories, customers, or teams is a governance problem before it is a capability. Memory that persists without a documented scope is memory nobody can audit.
Approval and Escalation Paths
Approval is where the layer decides that a human is required, and current agentic-AI governance guidance frames this as significant checkpoints or action boundaries that require human approval. This is the responsibility most often implemented as a UI feature rather than a control.
Four things need definition:
- Trigger — what conditions require approval, expressed in terms the layer can evaluate rather than the model's own judgment
- Recipient — who is asked, and what happens when they are unavailable
- Timeout behavior — whether waiting blocks, proceeds, or fails, and which is the safe default for that action class
- Record — what is written down when a human approves or overrides
The failure mode worth naming: approvals that exist but can be disabled by selecting a different execution mode. GitHub's protected-branch model makes the distinction concrete: required reviews still need an explicit non-bypass setting to cover privileged roles. A gate that the run can turn off is a gate only for the people who did not read the settings.
What Does Not Belong in the Layer
Boundaries are defined as much by exclusions, and this list prevents the layer from becoming the whole system.
Domain and business logic. If the rule is about your product rather than about controlling agents, it belongs in your application.
Task decomposition and prompt content. How to break down a problem is the agent's job. A layer that dictates reasoning steps has become a workflow engine with extra steps.
Judgment about code quality. The layer can require that a review happens. It cannot supply one.
Model capability. Routing decides which model runs; it does not change what that model can do.
Commercial packaging. Billing, seats, and tenancy are platform concerns. Keeping them out is what makes the control plane portable.
Architecture Risks for Coding Workflows
Five risks, each traceable to an unclear boundary.
Boundary drift. Permissions start in the layer and migrate into prompts as teams work around friction. The system still looks controlled, but enforcement has moved somewhere unenforceable. Audit this on a schedule — it happens gradually and nobody announces it.
Unscoped state. Memory or context crossing repository or customer boundaries because no scope was defined. Usually discovered when something appears in the wrong output.
Silent routing changes. Availability-based fallback that changes which model ran without recording it invalidates every comparison drawn across that period.
Bypassable approval. Covered above, and the most common gap between an architecture diagram and the running system.
Log gaps. If you cannot reconstruct why a step happened — which model, which tools, which state, whose approval — you cannot debug the system or answer an auditor. This is a layer responsibility precisely because no single agent has the whole picture.
The pattern underneath all five: the layer's value is that it makes control explicit and inspectable. Every workaround that moves a control back into a prompt, a default, or an agent's discretion trades that away for convenience.
Designing where those controls sit — parallel execution, plans reviewed before they run, approval paths that survive the run — is the category Verdent works in, and it is a design question worth settling before adding more agents to a system.
FAQ
Who audits old routing decisions after migration?
Whoever owns the layer — but the practical answer depends on what the migration preserved. Routing records are auditable only if they were written to a store that outlives the routing configuration itself; logs held in a vendor's console typically do not survive leaving that vendor. Before migrating, export the routing history you would need to interpret past evaluations, and name someone responsible for it afterward. If nobody owns the old records, the honest position is that pre-migration results are no longer interpretable — better stated than quietly assumed.
How should vendors expose layer-level logs?
At minimum each step should be reconstructable: which model and settings, which tools were available and which were called, what state entered and left, and which approvals or overrides occurred, with timestamps and identities. Export matters as much as display — a log you can only read in a vendor UI cannot be joined with your own incident data. Ask for machine-readable export, a documented retention period per record type, and whether deleting a project deletes its logs. Those three answers decide whether the logs are usable for audit or only for demos.
Can separate products share one control layer?
Yes, and it is often the right design, but scope has to be explicit at three points: durable memory, credentials, and approval routing. Shared routing logic and shared tool registries transfer well. Shared memory usually does not — it is the fastest way for one product's context to appear in another's output. The workable pattern is a common control plane with per-product scopes that are enforced by the layer rather than by convention, plus a test that verifies isolation rather than assuming it.
Which incidents need external stakeholder updates?
The NIST AI Risk Management Framework treats communication to relevant AI actors and affected communities as part of incident handling. For this layer, that means anything that crossed a boundary it was supposed to enforce: an agent reached a resource outside its intended scope, data appeared in an output where it should not have, an approval was bypassed, or a change landed without the review your process promised. Internal degradation — a slow run, a failed step, a routing fallback — is usually operational. The distinction is whether the failure affected something you told someone else you controlled. Decide this mapping before an incident, because the pressure of an active one is a poor time to define disclosure thresholds.
How should teams sunset experimental controls?
Give every experimental control an owner and a review date when you add it, and treat the absence of either as a reason not to add it. At review it either becomes a documented default or is removed — leaving it as a per-run flag is how a system accumulates options nobody understands. When removing, check what silently depends on it: experimental controls often become load-bearing without anyone deciding they should be. Remove in a low-risk environment first and record the removal, because "we used to have a toggle for that" outlives the toggle.
Related reading
- Once the layer runs across a team, the next question is who owns an agentic workflow
- For the approval gate specifically, automated review works better as queue triage than as an approval bot
- On making intent reviewable before code exists, see spec driven development for AI coding teams
- For a harness that pushes most of these controls out to extensions, see Pi coding agent
- And before adopting any of it against a real repository, work through this pre-adoption checklist
