How we chose a sandbox for our coding agent, and why we're retesting Cloudflare

Written with Evan Youngblood, Founding AI Engineer at Introzy, who ran the evaluation and the spike described here.

Atlas is our internal coding agent. Give it a small, well-scoped Linear ticket and it returns a draft pull request against our monorepo. To do that it needs a disposable Linux machine running the whole introzy-core dev stack: a Next.js dev server, Postgres, Redis, and a headless browser.

In July we spent two days choosing where that machine would live. On paper, Cloudflare's Sandbox SDK fit our security requirements best, because it let us enforce network and credential policy in code that runs outside the sandbox. During the spike we couldn't get our container to start, and with a deadline close we shipped on exe.dev instead.

Since then Cloudflare has rebuilt Containers for agent sandboxes, and we're planning to test it again. Treat the scorecard below as a record of July 2026. Two of the three vendors have shipped real changes since, and where the docs have moved, we say so.

Measure the workload before reading pricing pages

Pricing pages compare vCPUs and RAM. Our first job was finding out how much we needed. Evan ran our full verification tier (typecheck, unit tests, the dev server, and a Playwright pass) and sampled memory:

The dev server is 74% of the total. Most of a coding agent's memory goes to the app it's working on. That one measurement settled a question early: an 8 GB ceiling was out, whatever else a provider offered.

The bar: assume the agent is already compromised

The agent reads ticket text, repository files, dependency READMEs, and test output. All of that is untrusted input, and indirect prompt injection means any of it can carry instructions. Simon Willison's lethal trifecta names the failure: an agent with access to private data, exposure to untrusted content, and the ability to communicate externally can be tricked into handing that data to an attacker. A coding agent has all three out of the box, and the private data in our case is the source of a multi-tenant product that holds customer referral and finance data.

So we wrote the requirement for the worst case:

A fully compromised agent cannot reach anything except allowed hosts and cannot hold real credentials.

That sentence bundles two separate controls, and they're worth keeping apart:

• Reachability: which hosts the machine can talk to. That's egress policy.
• Authority: what the machine can prove it's allowed to do. That's where credentials live.

A hostname allowlist can say the sandbox may talk to github.com. It can't say whether the sandbox may push to main. Only something outside the sandbox that holds the credential and understands the request can enforce that. Anthropic reached the same conclusion building Claude Managed Agents: "The structural fix was to make sure the tokens are never reachable from the sandbox where Claude's generated code runs."

The scorecard

We scored Cloudflare's Sandbox SDK (version 0.12), Daytona, and exe.dev against ten requirements. Four of them decided it:

Some notes on fairness to each vendor:

• Daytona was stronger on egress than our first draft of the comparison gave it credit for, and we corrected that in our own notes before deciding. What ruled it out was memory and the isolation class. Its isolation docs now list full VM sandboxes alongside containers, and its network limits describe strict per-sandbox allowlists on its higher tiers. Scored today, that column would look different.
• exe.dev stores integration secrets server-side, never visible from the VM. The agent can't read the token. It can use the integration, because the proxy endpoint answers from inside the guest. That's a capability without a credential, a distinction we'll come back to next week. In July its HTTPS exec API also had a 30-second limit, so we'd have to drive it over SSH.
• Cost was a wash. Roughly $0.52 per two-hour run on Cloudflare and $0.66 on the others. Model spend dwarfs both.

What put Cloudflare at the top of the scorecard was one primitive. An outbound handler is Worker code that sees every HTTP request leaving the container, and it runs outside the sandbox. Cloudflare's docs show the basic pattern:

export class MySandbox extends Sandbox {}
MySandbox.outboundByHost = {
  'github.com': (request: Request, env: Env, ctx: OutboundHandlerContext) => {
    const requestWithAuth = new Request(request);
    requestWithAuth.headers.set('x-auth-token', env.SECRET);
    return fetch(requestWithAuth);
  },
};

In their words: "No token is exposed to the sandbox. The secret lives in the Worker's environment and is never passed into the sandbox."

Because the handler is code and not a list, it can read what's being asked for. Our plan was to inspect each git push and refuse anything not aimed at an atlas/* branch. A smart-HTTP push opens with pkt-lines of the form <old-sha> <new-sha> <ref>, so the check is short:

// Sketch of the plan. The spike ended before we ran it.
'github.com': async (request, env) => {
  if (new URL(request.url).pathname.endsWith('/git-receive-pack')) {
    const body = new Uint8Array(await request.clone().arrayBuffer());
    if (pushedRefs(body).some((ref) => !ref.startsWith('refs/heads/atlas/'))) {
      return new Response('push refused', { status: 403 });
    }
  }
  const authed = new Request(request);
  authed.headers.set('authorization', `Basic ${btoa(`x-access-token:${env.GITHUB_TOKEN}`)}`);
  return fetch(authed);
},

// pkt-line: 4 hex digits of length (including themselves), then the payload.
// "0000" ends the ref-update list; the packfile follows.
function pushedRefs(body: Uint8Array): string[] {
  const text = new TextDecoder();
  const refs: string[] = [];
  for (let i = 0; i + 4 <= body.length; ) {
    const len = parseInt(text.decode(body.subarray(i, i + 4)), 16);
    if (!len) break;
    const line = text.decode(body.subarray(i + 4, i + len)).split('\0')[0].trim();
    refs.push(line.split(' ')[2]);
    i += len;
  }
  return refs;
}

A production version needs more: a cap on buffered body size, handling for compressed request bodies, and a test for every refused case. Even so, no other candidate let us put policy at that layer, and GitHub can't do it for us, because installation tokens are scoped to repositories and permissions, not branches.

The spike: our container wouldn't start

Deployment went smoothly. The Worker answered /health, bearer auth rejected bad tokens, secrets were set, and the image was in the registry (4.17 GB uncompressed, about 1.2 GB compressed). Then the first real request:

POST /v1/sandbox/:id/session
502  pool error: Failed to start container        (~1.4 s)

It returned the same result on every retry. We checked five things:

The speed of the response suggested the start was rejected before scheduling, which points at something specific to our account or image rather than a container crashing. Two quick tests were left: switching the instance type from standard-4 to basic, and forcing a fresh push of the image. Each takes about ten minutes. We didn't run them, because by then exe.dev was working and we chose to ship.

Why we stopped debugging and changed the bar

We had a deadline and a working alternative. So Evan asked a different question: was the bar that picked Cloudflare the right bar for version one?

Atlas v1 is narrow on purpose. A human triggers every run. One repository, one run at a time, small tickets only, draft PRs only, and a person merges. The VM is destroyed after every run. Inside those limits, Evan proposed moving egress allowlisting from launch blocker to defense in depth, and wrote down exactly what that cost:

With open outbound egress, a fully compromised or prompt-injected agent can exfiltrate the introzy-core clone and any in-VM material, and can make arbitrary outbound calls for the lifetime of the VM. Mitigations bound the blast … they do not prevent it.

The next line in his note was "Not scary; honest." We think every accepted risk should be written down that plainly, with what it costs and what still holds.

Some things didn't move. No production or staging credentials in the VM. A model key with a hard spend cap. A GitHub credential the guest can't read. The VM destroyed after every run. A human review before merge. One control was demoted to "strongly prefer": in-guest iptables rules that drop private address ranges. Root inside the VM can remove them, but they turn "accidentally reachable" into "deliberately bypassed." We meant to block the cloud metadata address too, then learned exe.dev's integrations answer on it, so that one stays open.

Relaxing the bar also reopened the field. Evan's note says exe.dev "should win an audit, not a mood," and named Modal and E2B as the candidates to re-check under the new rules.

The dress rehearsal

exe.dev passed a full dress rehearsal on July 8 and got the go on July 9. The numbers:

Creating the VM takes about a second. Getting to useful takes more than two minutes, spent installing Postgres, Redis, Playwright, and dependencies. Startup benchmarks measure the first number, and your agent waits on the second. If you compare providers, compare time to the first useful tool call.

Evan tore down the Cloudflare deployment that evening, with notes on what to try next time.

What Cloudflare has released since

On September 30, Cloudflare rebuilt Containers for agent sandboxes. Three of the changes line up with what we'd want from a second attempt.

You own the Durable Object now. Under the new durable_object scheduling policy (public beta), your own Durable Object picks the image and instance type when it calls ctx.container.start(). Cloudflare's explanation is that hiding the Durable Object behind the Container class "made it harder for you to see and combine the identity, state, and coordination it provides." For us, that puts our own code at the point where the container starts, which is exactly where we'd want visibility on a retry.

Startup is much faster. On ComputeSDK's burst benchmark, median time to interactive dropped from 4.049 s to 648 ms and p95 from 5.839 s to 910 ms. Cloudflare also reports starting 100,000 containers in 5.4 seconds across six locations.

Sandbox SDK 1.0 is a set of helpers inside your Durable Object, not a base class you extend. Outbound handlers are per hostname. S3Mount signs R2 requests from the Worker, so the bucket credential never enters the container. The 0.x line we tested gets fixes until December 31, 2026.

We're not the only ones who want this shape. Cursor now runs its cloud agents' tools on Cloudflare Containers: Cursor runs the agent loop, and the container does the work. It's the same split between brain and hands that Anthropic describes.

We don't know yet whether any of this changes our July result, and a benchmark median doesn't predict our time to ready. That's what the retest is for.

What we'd do differently

File the ticket anyway. We had the evidence written up, down to a paste-ready support ticket, and didn't send it once exe.dev worked. Cloudflare's team could likely have told us what was wrong in a day. Next time, the ticket goes out the day we park the problem.

Run the cheap tests before walking away. Stopping the investigation was right. But switching to basic and re-pushing the image were ten minutes each, and running them would have closed the record.

Make observability a requirement. Our ten requirements covered egress, credentials, isolation, size, sessions, and cost. None asked, "When it fails, does it tell you why?" We now ask every provider for an operation ID on every start, timestamps per stage (auth, scheduling, image pull, boot), and SDK errors that keep the platform's message.

Measure time to ready, not time to VM. The 0.4-second number was nice to see, but the agent waited on the 133-second one.

What we're testing next

We're planning the retest on Sandbox SDK 1.0 with the durable_object scheduling policy. The plan:

1. Boot the same image with the new policy, on standard-4 and on basic, from a fresh push.
2. If the start fails, file the support ticket the same day, with the operation details the new setup exposes.
3. If it starts, build the git-receive-pack handler above and test every refused case.
4. Measure time to ready against exe.dev's 133 seconds, since that's the number the agent waits on.

If it passes, the question becomes whether programmable egress is worth a migration now that Atlas runs without it. We'll publish what we find either way.

Next week we'll cover the rest of Atlas's safety model: what it can touch, what it can't, and the dependency incident that changed it. The same idea, keeping consequential decisions out of the model's hands, runs through how Kai's deal agent writes to the database and how we check AI-generated code.

We'll be at Cloudflare Connect in San Francisco, October 19–21, including the sandbox egress session on Tuesday morning. If you work on Containers or the Sandbox SDK, we'd like to compare notes before the retest.

We ship in public. Come build with us.

Introzy is free to start. No card required.