Multi-Tenancy for Agent Workloads

Two business units share the platform. One of them ships an agent with a runaway reasoning loop that burns the month's token budget in an afternoon. The other one should never have noticed.

You have built multi-tenant systems before. The instincts transfer partially, and the part that does not transfer is exactly the part that causes the outage, because agent workloads violate an assumption sitting underneath every multi-tenancy model you have used.

The problem

Conventional multi-tenancy assumes a request does a bounded amount of work. You do not know precisely how much CPU a request will use, but you know it is one request's worth, and you size and throttle accordingly. Request rate is therefore a usable proxy for load, and per-tenant rate limits are a usable control.

Agents break that assumption in both directions at once.

One request is unbounded work. A single user message can produce a reasoning loop of forty model calls, twelve tool invocations, and three delegations to other agents, each of which starts its own loop. There is no fixed relationship between requests admitted and work performed. A per-tenant limit of 100 requests per minute constrains nothing useful.

The scarce resource is not yours. In a normal service, the contended resource is capacity you own and can add. In an agent platform, the resource that actually runs out is provider quota — tokens per minute on a model deployment, requests per minute on a rate-limited endpoint. You cannot scale it by adding nodes, it is enforced by someone else, and when tenant-a exhausts it, tenant-b receives 429s from infrastructure neither of them controls.

That combination produces failure modes classic isolation models do not anticipate:

• Token burn. A loop that fails to terminate consumes budget at machine speed. The damage is financial and immediate, and unlike CPU it does not self-limit — there is no point at which the system gets slow enough to stop the bleeding.
• Tool call storms. An agent retrying a failing tool hammers a shared MCP server and, through it, a downstream system of record with its own capacity limits and its own on-call rotation. The blast radius extends past the platform entirely.
• Context bloat. Agent memory grows without an obvious ceiling. One tenant's episodic retention policy silently becomes everyone's storage cost and everyone's index latency.
• Recursive amplification. Agents can invoke agents. A three-level delegation chain with a fan-out of four at each level is sixty-four leaf tasks from one user message. Traditional tenancy models have no concept of a request that recruits more requests.
• Sticky capacity. Sessions with affinity cannot be rebalanced away from a hot replica the way stateless traffic can. Module 2.4.

How it works

Start with the tenancy model, then the isolation planes.

Shared everything. One control plane, one model gateway, one index, a tenant identifier on every record. Cheapest to run, highest utilisation, adequate for internal tenants with similar risk profiles. Its weakness is that isolation is entirely a matter of correct code — one missing tenant predicate on a retrieval query is a cross-tenant data leak.

Shared control plane, isolated data and quota. The common landing point. Registry, gateways, and policy engine are shared; knowledge indexes, memory stores, and provider quota allocations are per-tenant. You keep most of the utilisation benefit and move the highest-consequence isolation — data and spend — into infrastructure rather than code.

Dedicated stack. Separate deployments per tenant, sometimes separate provider accounts and subscriptions. Reserved for regulatory separation or genuinely hostile multi-tenancy. Utilisation is poor and operational cost multiplies by tenant count, so it should be a deliberate answer to a specific requirement, not a default.

Whichever model you pick, five planes need explicit treatment:

The enforcement points for the two that matter most are the model gateway and the action gateway, which is convenient: both were already mandatory for other reasons, so tenancy controls ride on infrastructure you were building anyway.

A workable quota shape, expressed as configuration rather than code:

# Per-tenant limits evaluated at the model gateway, pre-flight.
tenants:
  - id: tenant-a
    model_quota:
      reserved_tokens_per_min: 200000   # guaranteed floor, never lent out
      burst_tokens_per_min: 600000      # ceiling when the shared pool is idle
      monthly_budget_usd: 40000
      on_budget_exhausted: throttle     # throttle | block | alert_only
    concurrency:
      max_in_flight_tasks: 200          # the control that actually bounds work
      max_agent_depth: 3                # delegation depth cap; stops recursive fan-out
    action_limits:
      writes_per_min_per_tool: 60
  - id: tenant-b
    model_quota:
      reserved_tokens_per_min: 200000
      burst_tokens_per_min: 400000
      monthly_budget_usd: 15000
      on_budget_exhausted: block
    concurrency:
      max_in_flight_tasks: 80
      max_agent_depth: 2
    action_limits:
      writes_per_min_per_tool: 20

Three things in that shape do the real work. The reserved floor is what makes the opening scenario a non-event: tenant-b's 200k tokens per minute are never lent to tenant-a, so tenant-a's runaway loop can consume the entire burst pool and tenant-b still gets its guaranteed allocation. The in-flight task cap bounds work rather than arrivals. The delegation depth cap has no analogue in conventional multi-tenancy, and it is the difference between a bad afternoon and an exponential one.

Platform design implications

Meter at the gateway, in the request path. The gateway already sees every prompt and every completion, so it already knows the token counts. Decrement synchronously and reject when exhausted. It is the only place where the decision can be made before the money is spent.

Prefer concurrency limits to rate limits. A limit of 200 in-flight tasks is a real bound on damage: a runaway agent occupies one slot and either finishes or is killed. A limit of 1,000 requests per minute is not, because one admitted request can generate arbitrary downstream work.

Attribute at the deepest level, aggregate upward. Every model call and every action carries tenant, agent, session, and task identifiers. Cost attribution and chargeback become queries rather than an estimation exercise. Module 5.5 covers the propagation mechanics; the tenancy requirement is simply that the tenant identifier be non-optional.

Make the tenant part of the credential. If tenancy is asserted by the caller, some path will assert the wrong one. If it is a claim in the agent's credential, verified at the gateway, cross-tenant access requires forging a credential rather than passing a wrong string.

Give tenants their own alerting. A tenant approaching its budget should hear it from the platform before being throttled. Platforms that throttle silently generate incidents in someone else's team, which is how a shared platform loses tenants.

Tradeoffs and decision framework

Isolation versus utilisation. Reserved floors are unused capacity most of the time. That is the price of the guarantee, and it is usually worth paying on the reserved portion while letting the burst pool be genuinely shared. Set reservations from observed p50 usage, not from peak.

Per-tenant provider deployments versus one shared deployment. Separate deployments give true quota isolation and clean per-tenant cost attribution, at the cost of idle capacity and more deployments to manage. A small number of high-value tenants: separate. Many small internal tenants: share the deployment, isolate with gateway-side quota.

Throttle versus block on budget exhaustion. Throttling degrades gracefully and keeps critical agents alive; blocking is unambiguous and prevents surprise overspend. The right answer usually differs by tenant and by agent criticality, which is why it belongs in configuration rather than in the platform's code.

How hard to isolate knowledge. Per-tenant indexes are the safe answer and multiply operational cost by tenant count. Shared indexes with enforced tenant predicates are cheaper and leave you one query bug away from a cross-tenant disclosure. If any tenant is external, or the data is regulated, take the operational cost.

Common mistakes

• Rate limiting requests instead of bounding concurrent work. The admitted request is not the load; what it recruits is.
• No delegation depth cap. Agent-to-agent recursion turns a linear workload into an exponential one, and nothing in a conventional tenancy model looks for it.
• Enforcing budgets from provider billing data. Lagging by minutes to hours, several orders of magnitude slower than a loop spends.
• Sharing one provider deployment across tenants of different criticality. One tenant's spike becomes another's 429s, invisibly, because the failure is enforced outside your system.
• Tenant identity passed as an argument rather than carried on the credential. Guarantees that one code path eventually passes the wrong one.
• Filtering retrieval by tenant in the caller. The retrieval service must enforce it. Anything a caller can forget, a caller will forget.
• No reserved floor. Purely shared burst capacity means the best-behaved tenant suffers first.
• Forgetting that memory and traces are per-tenant data. Retention, residency, and deletion obligations apply to agent memory and reasoning traces exactly as they do to primary data. Modules 2.3 and 10.2.

The through-line: bound work and spend rather than arrivals, make tenancy a property of identity rather than a parameter, and meter where the money is spent rather than where it is reported.