The Swarm Problem: How Vinkius Gateway Coalesces 100 Concurrent Agent Sessions Without Losing State

The Swarm Problem: How Vinkius Gateway Coalesces 100 Concurrent Agent Sessions Without Losing State

I watched a customer deploy twelve AI agents onto a single supply chain optimization task last month. By day three the agents were deadlocking on shared inventory state, racing for supplier API rate limits, and creating cascading invoice revisions that no single agent could unwind. The customer's conclusion was that the agents were "too smart for their own good."

That is not what happened. The agents were not too smart. They were too alone. Each one thinking it owned the session, each one writing state that the others would never reconcile, each one burning through the same pooled rate limit without knowing its peers existed. The problem was not agent intelligence. It was the session layer they all shared, and the fact that nobody owned it.

This is the swarm problem. You do not get one agent anymore. You get a crew, a squad, a swarm. Finance agents, inventory agents, procurement agents, compliance agents. All talking to different services through different credentials, all sharing the same rate budget, all believing the state they read five calls ago is still valid. The old model assumed one agent, one session, one lifecycle. The new model assumes none of those things.

Vinkius solves this with a gateway pattern, not an agent pattern. The gateway is the session owner. The agents are guests. The numbers that govern this design are not aspirational. They are extracted from the runtime config and the SSRF guard and the circuit breaker code, each one derived from an external constraint. Here is how the math works.

The B2BUA That Owns the Session

The SwarmGateway is a Back-to-Back User Agent. Not a proxy. Not a load balancer. A B2BUA: it terminates the inbound MCP session from the caller and establishes a separate outbound session to each specialist agent. The gateway speaks for the caller, but the caller's session ends at the gateway.

The gateway is configured with a registry that maps specialist names to upstream URLs, a shared delegation secret, and a set of operational limits. When an agent triggers a handoff, the gateway mints a scoped delegation token with an HMAC signature. The token carries claims: issuer, subject, issued-at, expiry, target agent ID, optional carry-over state, and a W3C trace context identifier so the specialist can correlate back to the originating trace.

The token TTL is sixty seconds. That is the constant in the code: tokenTtlSeconds = 60. It is not arbitrary. Sixty seconds is the window where a handoff completes its handshake, the specialist authenticates through the gateway, and the return-to-gateway path is established. Longer than that and a stolen token stays valid past the attention span of any single agent turn. Shorter and legitimate handoffs get killed mid-flight by network jitter.

The idle timeout is five minutes. idleTimeoutMs = 300_000 in the gateway configuration. When a delegated session sits idle beyond five minutes, the gateway evicts the transport, closes the upstream socket, and reclaims the slot. The sweep interval is fifteen seconds, so the gateway knows within that window that a session has gone dark.

The session cap is one hundred. maxSessions = 100. This is the ceiling on concurrent delegated sessions the gateway will track before it starts rejecting handoffs with a machine-readable refusal. Beyond one hundred, the gateway stops minting tokens and returns an error that the calling agent can read and act on.

The connect timeout is five seconds. connectTimeoutMs = 5_000. When the gateway tries to reach a specialist agent upstream, it has five seconds to establish the connection before it aborts and rolls back the handoff. This is deliberately tight: a slow specialist should fail fast, not hang the swarm.

All of these limits propagate through the V8 isolate boundary. The dispatch budget is thirty seconds, defined in Limits.ts as DISPATCH_TIME_BUDGET_MS = 30_000. The boot budget is five seconds, BOOT_TIME_BUDGET_MS = 5_000. The heap cap is one hundred twenty eight megabytes, ISOLATE_MEMORY_LIMIT_MB = 128. When a tool call inside a delegated session exceeds any of these, the TimeoutClassifier steps in to determine whether the breach was upstream I/O wait or host-side compute, and the error message carries a recovery hint to the calling agent.

Session Isolation: Fifty Per Token, Two-Minute Idle

The gateway does not own session lifecycle alone. The SessionManager, which sits in the runtime layer, enforces two session caps in parallel.

The first cap is fifty sessions per token. MAX_SESSIONS_PER_TOKEN = 50 in the SessionManager code. This is the guardrail that prevents one compromised or misbehaving connection token from exhausting the entire session table. When a token reaches fifty active sessions, the next connection attempt is rejected with a clear error rather than silently evicting an older session.

The second cap is time-based. The idle timeout is two minutes in normal operation, SESSION_IDLE_TIMEOUT_MS = 120_000. Under memory pressure it drops to thirty seconds, SESSION_PRESSURE_TIMEOUT_MS = 30_000. The sweep runs every fifteen seconds, SESSION_SWEEP_INTERVAL_MS = 15_000, so a session that goes quiet is reclaimed within one sweep cycle of its timeout.

Session metadata is replicated through Redis with a time-to-live of one hundred fifty seconds, REDIS_SESSION_TTL = 150. This is the mechanism that enables horizontal scaling: when a request arrives at a runtime instance that does not hold the session transport in memory, the route handler queries Redis for the session token and rehydrates the MCP server on the spot. The Redis TTL is intentionally set to match the idle timeout plus a buffer, so stale sessions expire automatically even if the sweeping instance crashes.

Memory pressure thresholds are defined as fractions of the container limit. MEMORY_WARN_THRESHOLD = 0.65 triggers warnings in the logs. MEMORY_PRESSURE_THRESHOLD = 0.80 triggers aggressive cleanup. The cleanup logic evicts idle sessions, forces garbage collection if the Node runtime exposes it, and logs the memory state so operators can see the pressure curve in real time.

The ConnectionTracker, which manages the transport-level cache, has its own idle sweep at thirty minutes, IDLE_TIMEOUT_MS = 30 * 60_000. This is the tier-one normal eviction path. Under memory pressure it has a second tier: when RSS crosses seventy five percent of the task memory budget, it evicts every cached config with zero active connections, oldest first, until pressure drops below sixty five percent. The code explicitly states: "This gives the auto-scaler time to provision new containers before OOM."

The coupling between these layers is deliberate. A session transport is not the same as a connection cache entry is not the same as a DNS-pinned outbound agent. They each have their own owner, their own timeout, their own eviction trigger. But they all decay on the same signal: the session went quiet, or the container is running out of memory.

State Sync: Causal Invalidation Without Stale Reads

The state sync layer is where multi-agent coordination stops being a hope and starts being a protocol. Vinkius implements causal invalidation with three mark types: immutable, volatile, and causal.

An immutable mark means the state is frozen. Once written, it cannot be changed. A volatile mark means the state is ephemeral and may be evicted at any time under memory pressure. A causal mark means the state is invalidated when any of its upstream dependencies change. These marks are not stored inside the V8 isolate. They are tracked in the host's state sync layer, which pushes invalidation signals through the Redis Streams pipeline.

When a specialist agent modifies a resource, the gateway publishes an invalidation event to the appropriate stream. Any agent holding a causal mark on that resource must re-resolve its state before the next tool call. This prevents the classic multi-agent problem where Agent Finance reads a price from Agent Inventory, Agent Inventory updates the price, and Agent Finance bills the customer using the stale value.

The carry-over state pattern from the SwarmGateway reinforces this. When the gateway mints a delegation token, it embeds the caller's intent as carry-over state in the token claims. The specialist receives this state and can act on it, but the gateway retains the authoritative copy. When the session returns to the gateway, the carry-over state is reconciled against the gateway's records, and any causal marks that the specialist touched are invalidated across the swarm.

The connection between sessions and the crypto audit trail is what makes this durable. The hash chain is constructed as raw_base64 || previous_hash || sequence_number and signed with Ed25519 each time. V8 never touches crypto directly. The StreamingDaemon is the single-threaded worker that reads from Redis Streams, forges the hash chain, signs with Ed25519, and dispatches to SIEM destinations. The session key rotates every twenty four hours, and state changes are checkpointed into Redis Streams so the chain survives a worker crash without gaps.

Connection Pooling: Four Thousand Domains, One Hundred Sockets

The SSRF guard enforces a paired cache: DNS resolutions and pooled outbound agents are coupled by lifecycle. The DNS cache holds a maximum of four thousand nine entries, DNS_CACHE_MAX_ENTRIES = 4_096. When the cache is full, the oldest entry is evicted. The agent pool holds a maximum of one hundred connections, AGENT_POOL_MAX = 100. When the pool is full, the oldest pooled agent is closed and its DNS entry is dropped in the same operation.

This coupling is the defense against DNS rebinding. The guard resolves DNS before the request, validates that the resolved IP is not in a private range, and then pins that specific IP into the undici Agent's custom lookup function. The SNI and TLS name remain the hostname, so certificate validation still works correctly. A rebinding attack that returns a different IP after the initial DNS resolution cannot redirect the socket, because the socket is pinned to the vetted address.

The private IP ranges blocked are: loopback (127 slash 8), Class A private (10 slash 8), Class B private (172.16 to 172.31 slash 12), Class C private (192.168 slash 16), link-local (169.254 slash 16), zero network (0 slash 8), and their IPv6 twins (double colon 1, FC00 slash 7, FE80 slash 10). These are tested against each resolved address before the connection is established.

The keep-alive timeout on pooled agents is sixty five seconds, AGENT_KEEP_ALIVE_TIMEOUT_MS = 65_000. This is deliberately set above the standard ALB idle window of sixty seconds so that pooled sockets survive between conversation turns without being silently closed by the load balancer. Undici's library default of four seconds would close sockets between every turn, forcing a cold TLS handshake on nearly every real-world call. The sixty five second timeout keeps the connection warm for the duration of a typical agent conversation.

The sweep that cleans idle agents runs every sixty seconds, coordinated with the ConnectionTracker's thirty minute idle sweep. When a pool entry is evicted, the DNS entry dies with it. Without a live connection policy, the vetted address is re-derived and re-vetted on the next use. The code comments state this explicitly: "a DNS resolution is cached exactly as long as we hold a pooled agent for that hostname, both die together."

The Circuit Breaker That Prevents Cascade

The circuit breaker is the financial guardrail that stops a runaway swarm from burning through budget. It is a sliding window counter implemented in Redis. The defaults are five thousand requests per five minute window, with a fifteen minute cooldown period. These numbers come from the governance dashboard configuration, not from the runtime code. The runtime code in CircuitBreaker.ts mirrors the PHP layer's CheckRequestQuota::checkCircuitBreaker method.

When a request arrives, the breaker checks if the circuit is already tripped by looking up a TTL key. If the key has a positive TTL, the circuit is open and the agent receives a machine-readable refusal string: "CRITICAL: Financial budget ceiling exceeded. DO NOT RETRY this request." The agent is instructed to direct the human user to the Vinkius Cloud console to approve resumption.

If the circuit is closed, the breaker increments a counter keyed by a deterministic window key: the account ID and the floor of the current time divided by the window size in seconds. If the counter exceeds the threshold, the circuit trips. The tripped state is stored as a Redis SETEX with the cooldown duration as the TTL, so it automatically resets after the cooldown period.

The breaker fails open. If Redis is unavailable, the check throws an error that is caught and logged, and the request is allowed to proceed. This is a deliberate design decision: a Redis outage should not block legitimate agent traffic. The tradeoff is that during a Redis outage, the circuit breaker is disabled and agents can exceed their budget. The code comments note this explicitly: "fail-open prevents blocking legitimate traffic."

When a session is evicted under memory pressure, the agent pool entries follow. The cascading effect is controlled: the breaker stops new requests, the session sweep removes idle sessions, the connection tracker evicts idle caches, and the DNS cache drops entries that no longer have live connections backing them.

The 34 Plus 4 Rules That Contain Each Session

The IsolateRunner enforces thirty four plus four engineering rules that keep each delegated session from escaping its sandbox. The thirty four rules are enforced at boot, snapshot, dispatch, and disposal. The four additional rules are memory, CPU, network, and disk containment.

At boot: a life-support polyfill replaces every Node builtin that could touch the outside world. The guest callback registry is registered before the IIFE executes. Binary-safe fetch is injected as a host bridge call, not a polyfill that the bundle can replace. The script is released immediately after boot, so the snapshot cache cannot hold a reference to boot-time objects.

At snapshot: V8 heap snapshots are cached to disk with a four-layer integrity model. The bridge stubs are replaced with host-owned references so the snapshot cannot capture a stale transport. State externalization hooks let the isolate shed references that should not survive a snapshot. The snapshot is keyed by deployment ID and invalidated atomically through Redis when the deployment is reimported.

At dispatch: structured clone with copy: true ensures that objects crossing from the isolate to the host are deep-copied, not shared. The dispatch timeout fires the watchdog, which kills the script even while it is parked on a host-side fetch. The TimeoutClassifier then determines whether the kill was upstream I/O wait or host-side compute, and the error carries a recovery hint.

At disposal: an AbortController wired through the entire dispose path aborts in-flight requests. Timer guillotine cancels every setTimeout and setInterval registered by the guest. References are released in reverse order, and a disposed guard prevents double-free. When a session is evicted, its in-flight requests are aborted in the same call, and the byte counter on the response stream is hard capped at ten megabytes.

The ten megabyte response cap is MAX_FETCH_RESPONSE_BYTES = 10MB in the runtime config. The byte counter is wired through the AbortController so a runaway response can be cut off mid-stream, not after it fills the isolate heap. The SSRF guard's safeFetch function enforces this counter on every outbound request, and the guard is the only path the isolate has to the outside network. There is no direct socket, no DNS-over-HTTPS tunnel, no WebSocket upgrade that bypasses the guard.

The four containment rules are: heap cap at one hundred twenty eight megabytes, enforced by V8's own resourceLimits.maxOldGenerationSizeMB flag. CPU is bounded by the thirty second dispatch watchdog, which is not a soft limit and cannot be extended from inside the isolate. Network is forced through the SSRF guard, which means the guard's DNS cache and agent pool are the only egress paths. Disk is the hard one: the isolate has no filesystem access at all. The bundle runs entirely in memory, and any file operations go through the host bridge.

Data Shielding Before the Agent Ever Sees It

Multi-agent sessions multiply the surface area for data leakage. One agent reads a customer record, another agent joins the conversation, and suddenly both agents can see the same sensitive field that only the first one was authorized to read. The DLP layer prevents this by operating before the data reaches any agent.

The ResponseGuard applies redaction patterns to every response before it leaves the gateway. The default patterns include wildcard matches for email, password, secret, credit card, SSN, phone, API key, token, date of birth, bank account, and IBAN. The wildcard syntax means *.email protects every email field at every depth in the response object, and items[*].credit_card protects array items specifically.

The redaction happens in RAM, never on disk, and never inside the V8 isolate. The guard runs in the host process before the response is handed to the MCP server. This means the sensitive data is masked before it enters any tool description or argument that an agent could read. The guard is stateless and deterministic, so the same response always produces the same redaction, which makes replay attacks on the audit trail impossible.

Each redaction is counted and attributed. The Security Posture report surface tracks the total redactions per hour, per connector, per agent session. When an agent triggers a redaction, the circuit breaker's error attribution knows it was DLP, not upstream, not the agent. The log entry reads Vinkius Err: DLP redaction applied and the recovery hint tells the agent to request filtered fields explicitly.

Tracing the Swarm: W3C Through the Handoff

When an agent hands off to a specialist, the W3C Trace Context travels with it. The traceparent header from the originating request is embedded in the delegation token claims, and when the specialist processes the request, it reads the trace context from the token and continues the trace.

The TraceContext helper in the runtime lifts the caller's trace context into the per-request context for every server factory. The context is present on all server types, API proxy, YAML, and bundle, so that tool spans and the request ledger can correlate a call back to the originating trace without any per-tool code. When the value is undefined, the trace is unrooted, not implicitly parented to a default.

The audit trail preserves the trace context end to end. The ChainForge constructs each audit record's hash from the raw base64 of the payload, the previous hash, and the sequence number. The trace context is embedded as searchable metadata in the log entry, so a security analyst can follow a single agent's path through the gateway, the specialist, and back to the gateway, all in one trace.

This is what makes the Live Activity table work. Each row shows the MCP server, the tool, the semantic action (query, mutation, destructive), the token, the outcome, and a full latency breakdown. Green means the upstream answered. Violet means Vinkius policy acted in flight. Amber means the caller erred. Red means the provider failed. Every color maps to a different failure owner, and every failure carries a trace that an analyst can open to see the full path.

The Return Trip: How the Gateway Re-enters

When a specialist agent finishes its work, the SwarmGateway's returnToGateway method activates. This is not a redirect. It is a state reconciliation. The gateway compares the carry-over state that was embedded in the delegation token against the state that the specialist returned, and any causal marks that the specialist touched are invalidated across the swarm.

The return trip is mediated by a tool that the gateway injects into the specialist session. This tool is not visible to the agent as a first-class capability. It is a return channel: the agent calls it with its final state, and the gateway takes over from there. The tool is branded with _MCPFUSION_handoff_return so the session layer recognizes it and activates the return path.

The gateway also rewrites the specialist's tool namespace. When the finance specialist exposes its tools, the gateway prefixes them with finance. so that the calling agent sees finance.create_invoice and finance.get_balance, not create_invoice and get_balance. This prevents name collisions when multiple specialists are active in the same conversation, and it makes the audit trail unambiguous: every tool call records which specialist namespace it came from.

The Numbers That Matter

Every limit in this system is named. There are no magic numbers hidden in configuration files. The derivation is documented next to each constant.

Name	Value	Source
Session idle timeout	2 minutes	SessionManager.ts, SESSION_IDLE_TIMEOUT_MS
Memory pressure timeout	30 seconds	SessionManager.ts, SESSION_PRESSURE_TIMEOUT_MS
Sessions per token cap	50	SessionManager.ts, MAX_SESSIONS_PER_TOKEN
Session sweep interval	15 seconds	SessionManager.ts, SESSION_SWEEP_INTERVAL_MS
Redis session TTL	150 seconds	SessionManager.ts, REDIS_SESSION_TTL
Memory warning threshold	65 percent	SessionManager.ts, MEMORY_WARN_THRESHOLD
Memory pressure threshold	80 percent	SessionManager.ts, MEMORY_PRESSURE_THRESHOLD
Connection idle eviction	30 minutes	ConnectionTracker.ts, IDLE_TIMEOUT_MS
DNS cache max entries	4,096	SsrfGuard.ts, DNS_CACHE_MAX_ENTRIES
Agent pool max	100	Limits.ts, AGENT_POOL_MAX
Agent keep-alive	65 seconds	Limits.ts, AGENT_KEEP_ALIVE_TIMEOUT_MS
Dispatch time budget	30 seconds	Limits.ts, DISPATCH_TIME_BUDGET_MS
Boot time budget	5 seconds	Limits.ts, BOOT_TIME_BUDGET_MS
Heap cap per isolate	128 MB	Limits.ts, ISOLATE_MEMORY_LIMIT_MB
Session key rotation	24 hours	StreamingDaemon.ts, SESSION_KEY_TTL_MS
Circuit breaker window	5,000 requests / 5 minutes	Governance config
Circuit breaker cooldown	15 minutes	Governance config

The swarm problem is not solved by making agents smarter. It is solved by making the session layer authoritative. The gateway owns the delegation token, the session lifecycle, the state sync, and the rate budget. The agent is the guest. The gateway is the host. When an agent overstays its welcome, the gateway evicts its session. When the pool is full, the gateway rejects the handoff. When the budget is breached, the circuit breaker trips and the swarm halts.

The post on AI agents as new consumers covered the MVA architecture: Model, Presenter, and Tools. The runtime post on V8 isolates covered the sandbox and the snapshot integrity model. The governance post covered the twelve surfaces and the circuit breaker. This post covers the layer they all depend on but never talk about: the session layer that prevents a hundred agents from stepping on each other. The next post in this series will cover the capability lockfile, and how the mcpfusion.lock file enforces breaking changes with git-diffable diffs and CI gates.

The numbers above are not my estimates. They are what the code enforces. You can read Limits.ts, SessionManager.ts, SsrfGuard.ts, and CircuitBreaker.ts in the cloud runtime. Every value is named, documented, and derived from an external constraint. That is the difference between a platform that scales and a demo that collapses.

The session management, quota enforcement, and circuit breaker that govern concurrent agents are answered with source code in Enterprise AI Questions.