Zero Cold Starts for Untrusted Code: How Vinkius Runs MCP Servers in V8 Isolates

Each MCP server a customer ships on Vinkius Cloud is, sooner or later, third-party code executing on our infrastructure. Not ours. Not a library we control. Theirs. A cloud for AI agents is only useful if customers can bring their own logic, their own connectors, their own glue between an LLM and their systems. Once that capability exists, the question no product deck answers becomes the whole engineering problem: how do you run untrusted JavaScript for thousands of tenants on a small number of servers without giving any one of them a path to hurt the others, the machine, or their neighbors?

This is the question behind the V8 isolate layer of the Vinkius runtime, and the part of the architecture I talk about the least in public, which is why I am writing it down here. What follows is how the system works today. The numbers are the ones the code enforces, not the ones I would like it to be.

Why not one container per server

The first answer any multi-tenant platform reaches for is a container per server. It is clean, it is understood, and it gives you real kernel-level isolation. I spent a serious amount of design time on exactly that.

It died on density. Vinkius Cloud customers connect MCP servers the way they connect integrations on any SaaS: dozens per workspace, cheap to create, cheap to delete, most of them idle for days. A Node.js process that has booted a real runtime (framework, SDK, polyfills, connection pool) carries a fixed cost of tens of megabytes before it does useful work. Multiply that by a few hundred servers on one task and you are fighting the task's memory ceiling before a single tool call has happened. Multiply that by cold starts, and you have built a platform where booting your own server takes hundreds of milliseconds.

The second obvious answer is many processes inside one container, one fork per tenant. It is worse. You trade the kernel boundary for nothing: each process still has its own heap, its own page tables, its own garbage collector, and a crash in one process can still take the siblings down through shared file descriptors and the init process. You spent the same memory as the container answer and got a worse failure mode.

I also ran the numbers on the middle ground: one Node process per workspace, shared across a tenant's servers. It falls on the same two walls. The fixed footprint of a booted Node runtime is a per-tenant cost whether you share it or not, and one workspace's memory leak or fork bomb takes out all the servers in that workspace with it. Blast radius does not localize.

What Cloudflare demonstrated with Workers is the third option: not a process per tenant, and not a container per tenant, but an isolate. V8 was built to run many untrusted programs inside one browser process, which is exactly the shape of our problem, because a browser tab is a tenant. The Workers runtime leans on the same primitive: an isolate per request, restored from a V8 snapshot, so the cold start people complain about is really just a heap restore. I did not adopt their platform. I cannot: our gateway is long-lived and stateful in ways a page function is not, and we host our own fleet on AWS where I control the upgrade cadence. I adopted the idea, and I built the host side of it on top of Node.js with isolated-vm, the C++ binding that lets a Node process own a raw V8 isolate.

That one decision ("a tenant is an isolate, not a process") is where the rest of the runtime comes from. What follows is a consequence of it.

What the isolate actually provides

An isolate is a heap: one garbage-collected region of memory with its own microtask queue, its own globals, and a hard cap you set at creation time. The isolate that isolated-vm creates for a tenant gets 128 MB. That is not a soft quota we enforce in application code and log a warning about; it is a memoryLimit handed to the engine. When a tenant's heap hits it, that tenant's isolate stops being served. The other tenants keep running. No tenant's globalThis is reachable from another tenant's isolate, because there is no shared JavaScript boundary at all. The only traffic that crosses the boundary is what we explicitly bridge, and we bridge almost nothing.

Scheduling cost matters more than people think. Spawning a process is a kernel event; forking takes page-table setup, an exec, and a garbage collector warmup. Switching isolates inside one process is a pointer swap. That is why Cloudflare can put the word "zero" next to cold starts, and why the latency floor of our edge-deployed servers is measured in milliseconds, not in the 200 to 400 ms range a fresh Node boot usually lands in.

The lifecycle layer states the isolation property more directly than I can: one runner per connection token, and no sharing between tokens. Credential isolation is absolute. The runner that owns a connection owns its isolate, its secrets snapshot and its timers, and nothing in the process graph reaches across it.

The trade is that an isolate is an empty environment. V8 does not ship a standard library. There is no fetch, no TextEncoder, no console, no setTimeout, no process, no Buffer. The first real work of the runtime is therefore not running tenant code. It is building the environment that tenant code runs in.

The empty environment

Booting a tenant isolate starts with what we call the life-support layer: one concatenated bundle of polyfills that restores the Web and Node API surface tenant code expects. The load order is deliberate, because each layer depends on the one below it. Console and process shims go first, since the polyfills above them log and inspect process. Then TextEncoder and TextDecoder with UTF-8 and surrogate-pair support, because emoji in a tool argument is not an edge case. Then URL and a full 12-method URLSearchParams (the changelog entry reads "axios, got, and @octokit now work correctly in edge bundles", and that was the point). Then crypto, timers, and the network stack: Headers, fetch, and an async-only XMLHttpRequest shim, so the old HTTP libraries people paste into connectors do not die of xhr is not defined.

The point is what is real and what is delegated. Nothing that touches the outside world exists inside the isolate. fetch in a tenant bundle is a polyfill on the guest side, but the actual request happens host-side, through __host_fetch, as a C++ ArrayBuffer copy across the boundary: binary-safe, so an image or a gzip response is not mangled into UTF-8 nonsense. crypto.getRandomValues is host-side randomBytes, capped at 64 KB. crypto.subtle.digest and HMAC, the ones JWT verification needs, are host-side too, and the host verifies signatures with a constant-time comparison. Timers are host-side setTimeout calls that invoke back into the guest through a registered invoker, capped at 30 seconds. The rule behind all of it: the polyfill makes the call look native, and the host owns the effect. A tenant can express intent. It cannot reach the operating system.

Arguments and results cross the boundary with structured cloning (copy: true in the isolated-vm API) instead of JSON.stringify. You pay the serialization once, in C++, and you keep real ArrayBuffers and typed objects on both sides. You do not notice this until you profile a tool call that moves a 1 MB payload: the JSON round-trip was the second most expensive part after the network.

On the compile side, tenant code never reaches the runtime as source. The bundle pipeline takes the server spec, generates the entrypoint, and runs it through esbuild (IIFE, browser platform, ES2022 target, minified), so what lands in the isolate is one flat file with no dynamic imports. A static-analysis sanitizer rejects the escape hatches before the bundle is accepted: direct __host_* bridge access, globalThis[...] bracket-notation sneaks, Function.constructor bypass patterns. Bundles are gzipped and SHA-256'd. The API allows up to 1.5 MB of raw bundle, and the runtime enforces a 2 MB decompression ceiling with a streaming byte counter that aborts before a GZIP bomb can fill the heap. One detail I stand by: the compiled bundle runs once in a disposable isolate with all bridges stubbed out, and that run is what extracts the tool manifest. The program is paid for at compile time, not at request time.

And here is the part developers never see. The bundle does not know it is in the cloud. The same startServer() call that boots a server over stdio on a developer's laptop detects the runtime's __vinkius_edge_interceptor global, hands over its tool definitions through that interceptor, and skips the normal transport setup. One codebase, two runtimes: you develop locally with the framework and deploy to the cloud without a single if (inCloud) branch.

Protocol versioning travels with the runtime, not the bundle. The gateway speaks the stateless MCP release of 2026-07-28 as its primary dialect and negotiates down through the 2025 release and the 2024-11-05 SSE transport as fallbacks, so a client a year old still talks to a server shipped this morning. The dialect a session ends up on is a handshake, not a deploy decision.

Why the guest never touches the network

The most dangerous thing a tenant bundle can do is make an HTTP request. fetch is a universal source of SSRF: a careless or malicious bundle pointing a tool at http://169.254.169.254/, the AWS metadata service, hands a customer's AI agent a read path into the instance's credentials. All outbound traffic therefore goes through the SSRF guard, and the guard works on one principle: an address is only valid for as long as the policy that approved it. It resolves DNS before the request and blocks private ranges outright (loopback, 10/8, 172.16/12, 192.168/16, link-local 169.254/16, which is the metadata service, 0/8, and the IPv6 twins). It then pins the approved IP into the undici connection, so a rebinding attack cannot swap the address after vetting. And it keeps SNI and TLS aligned with the pinned address, because pinning at the socket level without pinning the name would throw CERT_ALTNAME_INVALID on each request. The DNS cache is capped at 4,096 entries and lives no longer than the pooled connection it belongs to.

The keep-alive setting on that pooled agent is the most boring fix I have ever shipped, and I am glad to write it down. undici's default idle timeout is 4 seconds, and it was closing pooled sockets between conversation turns in the background, so nearly each real tool call paid a cold TLS handshake. We run it at 65 seconds, just above the ALB's 60-second idle window, and cap the pool at 100 sockets per target. The p50 of outbound calls dropped by the cost of one handshake. A boring fix with a p50 is the dream.

Responses stream into the isolate with a running byte counter, hard-capped at 10 MB, with an AbortController wired through the whole dispose path: when an isolate is retired, its in-flight requests are aborted in the same call. When a dispatch times out (the 30-second budget), the error is not a generic "request took too long". A classifier asks what the isolate was doing at the moment of breach: upstream I/O in flight, the guest computing, or memory pressure? The tool_error that goes back to the agent carries the answer, plus a recovery hint. Ownership of a failure (the tenant's upstream API, the tenant's code, or the platform) is not something you should figure out by grepping logs at three in the morning.

That classifier's output feeds an upstream-attribution ring with 32 slots, newest first. The operational picture then shows not only that dispatches are failing but whose upstream is. When a tenant's CRM API degrades at two in the morning, the runtime knows it is the tenant's CRM, not the platform, and the ring holds the evidence long enough for the incident channel to read it.

Provisioning once: the snapshot principle

A full boot (compile the bundle, run the polyfill layer, execute the IIFE) costs around 50 to 100 ms on first contact with a deploy. That is fine for request one. It is not fine for request 4,000, which is all of them.

The principle is that an environment and a program are different artifacts, and only one of them changes often. The polyfill layer is static for a given deploy: same V8 version, same host surface. What changes per request is the tenant code. So we snapshot the environment, not the program. The host runs the isolated-vm snapshot builder over the provisioned environment and caches the resulting heap blob on disk, keyed by deploy ID and stamped with the V8 version it was built under. On restore, a fresh isolate is created from that blob: a millisecond or two for the engine, another one or two to swap the stubbed bridges for real ones, and the IIFE itself re-executes in the restored context. Total: 15 to 25 ms, against 50 to 100 for a full boot, and the polyfill work happens exactly once per deploy. The bundle is re-run rather than snapshotted because Isolate.createSnapshot() is a static method that cannot execute async code, and our IIFEs are allowed to await at top level.

The hard part of snapshots is not the speed. It is the failure mode. A deserialization failure in V8 is a SIGABRT: the process dies, JavaScript cannot catch it, and a corrupt blob sitting on disk is a crash loop waiting to happen. Restore, abort, restart, restore the same blob, abort again. So the integrity model treats the cache like untrusted input. Four layers, in order: writes are atomic (the blob lands in a .tmp file and is renamed, metadata first, so a kill mid-write leaves a verifiably mismatched pair, not a half-blob); each blob is SHA-256'd and the hash is checked before the bytes reach V8; the V8 version stamp makes the cache self-invalidating the moment Node is upgraded on the fleet; and at process start a purge pass validates the cached snapshots on disk and deletes the bad ones. The design invariant underneath all of it: a corrupt cache entry degrades to a 100 ms cold boot. It never degrades to a crash. That is the whole incident: 80 extra milliseconds on one request.

The snapshot layer also doubles as hibernation. Stateful bundles expose getState and setState hooks with a one-second serialization budget, so a tenant that keeps a working set in memory can have its state extracted, its isolate retired, and its state injected back on the next request. Same machinery, reversed direction.

Invariants: what one phase owes the next

The layer above is only as honest as its invariants, and invariants rot when they live in people's heads. The runner therefore carries a written contract: 34 rules across its four phases (boot, snapshot, dispatch, dispose), next to the code they govern, read on each review that touches that file. The phases are the lifecycle; the rules are what one phase owes the next. Most of them are "no" rules. No bridge in a snapshot. No host timer outliving its isolate. No dispatch that skips timeout attribution.

Dispose is where the contract pays off, because disposal is the one phase where a wrong order leaks something. Retiring an isolate is a fixed five-step sequence: abort the AbortController first, so pending HTTP dies with the abort; clear the host-side timers, so the guest cannot reschedule itself; release the isolated-vm reference handles in reverse allocation order; release the context and the isolate; only then null the last JavaScript reference, so nothing pins a dead heap to the process. The concept is ownership transfer: no bridge may outlive the isolate it points at. Skip a step and you get a lingering heap, a timer that fires into a corpse, or a request that outlives its owner. The order is load-bearing, and the contract says so in writing.

Data-loss prevention gets the same treatment at the other end of the lifecycle. Before a payload reaches tenant code it passes a redaction pass that is loaded at boot, not lazily: a control that has not loaded yet does not exist. Credentials in transit and anything that looks like a secret are masked before they touch a log line or a bridge argument. A DLP inside tenant code is a DLP the tenant can turn off; a DLP on the host side of the bridge cannot be.

Containment by construction

The limits in the runtime are not a wall of magic numbers. They live in one file, Limits.ts, which documents, next to each constant, where the number comes from. The 30-second dispatch budget comes from user behavior: mainstream MCP clients give up on a tool call somewhere between 30 and 60 seconds, so beyond 30 we are spending resources on a result nobody reads. The 65-second keep-alive mirrors the ALB idle window. The sweep horizon belongs to the connection tracker's 30-minute idle tier. A limit is the footprint of a policy, not a constant: when the policy changes, one file changes, and the derivation stays on record.

Credentials follow the same discipline. A tenant's decrypted secret map is injected into the isolate as a deep copy on __vinkius_secrets: the guest reads it, it cannot write back to the host. The runner keeps a SHA-256 fingerprint of the map, not the values, which makes re-injection on a later request a cheap no-op when nothing changed. Each token gets its own runner and its own isolate; there is no pooling that shares credential state between tenants, full stop.

The only token shape that reaches an isolate is the live connection token, anything with the vk_live_ prefix. Preview tokens, revoked tokens, tokens for another deploy: they fail at the runner boundary, before a bridge is registered. The one capability the guest does hold is VINKIUS_TOKEN, exposed to the catalog's tool code: it authenticates a tool back to the platform as the user who owns it, scoped to that user's catalog. The model is capabilities, not passwords: each crossing of the boundary is a named, audited, revocable right.

What a bundle cannot do is documented as positively as what it can. No filesystem, no process.env, no direct bridge access (the sanitizer rejects it at compile time), 128 MB of heap, 30-second timers, 10 logs per second and 1 KB per message through the console bridge, a 10 MB fetch ceiling. When a tenant crosses the financial line, the circuit breaker does not return a 429 and shrug. The quota layer emits an AI-native error: plain language telling the LLM to stop retrying and to surface the budget ceiling to the human user, with a link to the Vinkius Cloud console where the owner approves resumption. A retry storm is the native failure mode of agents; the platform should answer the agent, not only the person behind it.

What the platform owns

All of this runs on a small, deliberately x86-only fleet. The x86 pin is a constraint I own in print: isolated-vm is a native addon, and its ARM builds have been the flakiest dependency we ship. On x86 it is boring, and boring is the goal in the layer where other people's code runs.

The process boots empty. No configuration is loaded at start; the first connection for a token pulls the server's configuration from the Laravel API on demand, boots that server's isolate just in time, and an LRU sweep retires entries after 30 idle minutes, so one task holds the live state of many tenants. Redis carries the control plane (invalidation, kill switches, tool-change notifications, quota updates), so a configuration change in the console reaches all tasks without a deploy.

One last piece of the security story sits next to the isolates, and I close on it because it shows the boundary we drew deliberately: the cryptographic audit path. Each tool call event flows to a single-threaded streaming daemon that forges a SHA-256 hash chain and signs it with an Ed25519 session key, held in RAM for 24 hours and certified by the master key on boot. After a crash the daemon reprocesses un-ACKed events, so the chain never has a gap. Each forged link is checkpointed to a Redis Streams consumer group, so a crash resumes from the last ACKed event instead of re-deriving the chain, and SIEM adapters drain that same stream behind their own circuit breakers: a failing sink gets fenced off rather than allowed to back up the audit path. The rule that made the daemon necessary fits in one line of source: V8 never touches crypto. Tenant code can compute a hash through the host bridges, but it cannot sign, certify, or forge anything. The audit trail is something the platform does to the runtime, not something the runtime's tenants can reach into.

Direction

The isolate layer today is the stateless, fast, hard-capped story: boot in a millisecond or two, cap at 128 MB, retire in a sweep, and let the snapshot carry the heavy part. The open direction is the stateful one: longer-lived tenant state riding the hibernation hooks, denser packing of live isolates per task, and eventually teams bringing their own connector catalogues into the marketplace without waiting on our CI.

I do not think V8 isolates are the final answer to multi-tenant edge code. They are the answer for the shape this platform has, and the shape is where I want it to stay: cheap tenants, hard caps, and a millisecond-scale boot, because an agent's patience is measured in seconds, and so is its trust. Cloudflare's "zero cold start" is a sentence you can engineer, not a slogan you can only admire. It is an isolate, a snapshot, an integrity chain, and a series of unglamorous decisions about who owns the timers. The code in this post is the runtime behind the MCP servers on cloud.vinkius.com, and the tracing work that ties its tool spans into distributed traces is in the MCP Fusion 5.1.0 post. The stateful direction this platform is moving toward is the session layer itself: how the SwarmGateway coalesces overlapping agent sessions and applies causal invalidation to stale state. The full architecture is in the SwarmGateway sessions post.

The V8 isolate sandbox and its 12-step boot sequence are covered with source code in Enterprise AI Questions.