The Reference Architecture
Before any of the detail, you need a picture you can defend in an architecture review. This lesson builds the whole platform on one page and everything after it fills in a box.
The three previous lessons gave you the object, the boundary, and the tenancy model. This one is the map. By the end you should be able to draw the platform from memory, trace a request through it, and name what each hop decides — which is precisely what an architecture review will ask you to do.
The problem
Most agent architectures presented in reviews fail on the same question: where exactly is that enforced?
The diagram shows an "agent layer," a "governance layer," and arrows. It looks complete. Then someone asks what happens when the agent decides to call the payments tool with an argument nobody anticipated, and the answer turns out to be that the governance layer is a set of libraries the agent teams import. That is not an architecture, it is a convention, and conventions do not survive contact with a model that generates its own plans.
A defensible reference architecture has a specific property: for every control you claim, you can point at exactly one component that enforces it, and explain why the agent cannot go around that component. Everything below is organised to make that possible.
The second thing reviews probe is the control plane / data plane split. Platform engineers know this instinct from Kubernetes and service meshes, and it transfers cleanly here — but only if you are disciplined about which decisions are made per-request and which are made once and cached. Get it wrong and your policy engine is in the hot path of every model call, adding tens of milliseconds to a request that already makes forty of them.
How it works
Six planes. Two of them are in the request path for every call; the rest are not.
Agentic platform reference architecture
Hover components for details
Now the path. A relationship manager asks an agent to raise a customer's credit limit.
Request path, with what each hop decides
Click each step to explore
Three things about that path are worth stating explicitly, because they are the parts reviews attack.
Identity is asserted once and propagated, never re-derived. The principal is established at step 1 and travels to step 6 intact. No component downstream guesses who the user is, and no component substitutes the agent's own identity for the user's. When the chain breaks, every control after the break degrades to "some agent did something."
Policy is evaluated in three places, for three different questions. Retrieval asks what may this principal see. The model gateway asks may this content reach this model. The action gateway asks may this specific action proceed. These are genuinely different questions with different inputs, and collapsing them into one policy service that answers all three is how you end up with a policy engine that answers none of them well.
Audit is written by the enforcing component, synchronously. Not by the agent, not by a log shipper. The only component that knows the policy decision and the identity chain at the moment of the action is the one that made the decision.
The test for whether your architecture is real: pick any control you claim to have, and name the single component that enforces it and the reason an agent cannot route around it. If the answer involves a library the agent teams import, or a rule in a prompt, or a convention documented on a wiki, that control does not exist — you have a recommendation.
Platform design implications
Keep the control plane out of the request path. Registration, policy authoring, budget setting, and routing configuration all happen out of band. The gateways hold compiled policy and configuration in memory, refreshed on a push or a short poll. This matters more here than in a typical service mesh because one user request makes dozens of gateway calls: a 20 ms policy lookup becomes 800 ms of added latency on a forty-step task, and that is the difference between a usable agent and an abandoned one.
A workable latency budget for the two in-path components:
| Hop | Budget (p99) | What it must not do |
|---|---|---|
| Model gateway overhead | 5–15 ms | Synchronous call to a policy service or registry per request |
| Guardrail inspection | 30–80 ms | Run serially with the model call when it could run concurrently |
| Action gateway decision | 10–30 ms | Fetch entitlements from a system of record per action |
| Audit write | 5–20 ms | Block the action on a slow analytics sink |
The pattern in the right-hand column is the same in every row: anything that needs a remote lookup per request must be pushed to the gateway ahead of time, cached with a bounded staleness you can defend, or made asynchronous where correctness allows.
Design the degraded modes before the happy path. For each in-path component, decide now what happens when it is unavailable, because you will decide it under pressure otherwise:
- Model gateway down: agents cannot reason. Full outage, and correctly so — the alternative is agents holding direct provider credentials, which defeats the architecture.
- Action gateway down: agents can reason but cannot act. Degraded, not down. Draft-only mode is a legitimate and useful posture.
- Policy distribution down: gateways keep serving the last compiled policy. Stale policy is far better than no enforcement, but bound the staleness and alert on it.
- Audit sink down: this is the one people get wrong. If audit is a compliance requirement, actions must fail closed when they cannot be recorded. An action you cannot evidence is, for a regulated institution, an action you should not have taken. Module 9.1 covers fail-open versus fail-closed generally.
- Knowledge foundation down: agents run ungrounded. Usually worse than failing, because an ungrounded agent answers confidently from parametric memory. Prefer failing the task to silently dropping grounding.
Retrieval that filters by tenant but not by the end user's entitlements is the most common serious hole in otherwise well-drawn architectures. The agent is acting for a specific principal; the index contains documents that principal is not cleared to read; the agent summarises one into an answer. Nothing in the action path is violated because no action was taken — the disclosure happened at step 3. Entitlement-aware retrieval is not a Module 6 nicety, it is part of the enforcement story.
Tradeoffs and decision framework
One gateway or two? They are drawn separately because they answer different questions with different data and different latency profiles, and because the action gateway usually needs to outlive several generations of model gateway. Small platforms often start with one process serving both roles, which is fine — keep the interfaces separate so the split is a deployment change later, not a rewrite.
Centralised policy service versus embedded evaluation. A central service is easier to reason about and gives you one place to audit; embedded evaluation with pushed policy bundles is what actually meets the latency budget above. The common answer is central authoring and distribution, local evaluation — the same shape service meshes settled on, for the same reasons.
Sidecar versus service for the gateways. A sidecar is unavoidable by construction if the network is configured so nothing else can egress, which is attractive. A central service is simpler to operate and upgrade, and easier to instrument consistently. Pick based on whether you can genuinely lock down egress in your environment; if you cannot, the sidecar's main advantage disappears and you are paying its cost for nothing.
How much to build before the first agent. Build the action gateway and identity first, even underpowered. They are the two that cannot be retrofitted, because retrofitting means finding and severing every direct path agents have already grown to real systems. The knowledge foundation, evaluation, and the tool registry can all arrive later without invalidating what exists.
Common mistakes
- Claiming a control that no single component enforces. The one question that reliably breaks an architecture review.
- Putting the registry or policy service in the request path. Multiplied by dozens of calls per task, it is the difference between a usable and an unusable agent.
- Re-deriving the user identity downstream. Assert once at the channel, propagate. Anything that re-derives will eventually derive the agent instead of the user.
- One policy engine answering all three policy questions. Retrieval, model access, and action authorisation take different inputs and have different latency budgets. Shared authoring, separate evaluation.
- Audit written by the agent. The agent does not know the policy decision, and it is the component whose behaviour you are trying to evidence.
- No defined degraded mode for the action gateway. Decided during an incident, it becomes "disable the gateway to restore service," which is the worst possible outcome.
- Retrieval scoped to the tenant but not the principal. A disclosure path that never touches the action gateway, so none of your action controls see it.
- Drawing the agent kernel with direct arrows to systems of record. If the picture has that arrow, so does the network, and the entire architecture is decorative.
Your reference architecture puts the action gateway in the path of every real-world effect. During an incident the audit sink becomes unavailable, and actions begin failing because they cannot be recorded. A senior engineer proposes making the audit write asynchronous and best-effort so actions keep flowing. In a regulated institution, what is the strongest counter-argument?
Carry the picture, not the boxes. Six planes, two of them in the request path, ten hops from user to effect, and one question you can answer for every control you claim. The remaining ten modules are each a deeper pass over one of these boxes, and they will assume you can place them on this map.
Draw the request path for a user asking an agent to perform an action against a core system. Name every enforcement point it passes through and what each one decides.