Zac Sheffer · 2026.10.19 · 10 min read
Written with Evan Youngblood, Founding AI Engineer at Introzy, who designed Atlas's execution model.
Atlas is our internal coding agent. It takes a small Linear ticket and returns a draft pull request against our monorepo. The companion post covers how we chose where it runs and why we're retesting Cloudflare.
Atlas is safe enough for its current job because its authority is narrower than its ability to write code. We don't rely on the model behaving, or on its report that the tests passed. The agent can propose a change. Code outside the agent decides whether that change gets published, checks it, and publishes it. A person decides whether it merges.
The end of the post lists what this design doesn't cover yet.
Every rule below assumes this scope. Change the scope and the rules need re-deciding.
Simon Willison's lethal trifecta is the clearest test we know for agent risk: private data, exposure to untrusted content, and the ability to communicate externally. If an agent has all three, an attacker who controls the content can get the data out.
| Ingredient | In Atlas | Status |
|---|---|---|
| Private data | Our source code, ticket context, prompts. The database is synthetic. | Present |
| Untrusted content | Ticket text, repo files, dependency code and READMEs, test output | Present |
| External communication | Open outbound network from the VM | Present, and an accepted risk |
We haven't broken the trifecta. All three legs are there. What we've done is limit what a successful attack gets. In the words of our own threat model: a compromised agent "may exfiltrate the checked-out introzy-core source, ticket context provided to the run, synthetic database contents, model prompts and responses." It doesn't get production credentials, customer data, deploy access, cloud admin access, or a way around branch protection. Those are never in the VM to steal.
The open network is a real hole, and the companion post explains how we came to accept it. Closing it is on the list at the end of this post.
We find it more useful to ask which principal holds which authority than to ask what the agent can do.
| Principal | Can do | Can't do |
|---|---|---|
| OpenCode agent, inside the VM | Read and edit the workspace, run commands, reach the network, call the model | Commit, push, choose the branch, open a PR |
| GitHub integration, in the VM | Push to the repository when called | Be read: its credential never enters the guest |
| Atlas control plane, outside | Run the checks of record, decide whether to publish, send the push, verify it, open the PR | Merge, rebase, force-push, bypass protection |
| A person | Review and merge | |
| GitHub | Enforce branch protection and required review on main |
We run OpenCode as the agent and configure it to deny git commit and git push. That setting is a convenience. The boundaries are the rows below it.
A capability is something a process can do from where it sits. A credential is a secret it could carry somewhere else. Atlas leans hard on the difference.
The GitHub integration is a capability. The VM pushes through exe.dev's integration, which keeps the secret server-side and never visible from the VM. A compromised agent can't copy that token out and use it somewhere else next week. That's the property Anthropic built into Claude Managed Agents, where the repo token is wired into the git remote at setup so "Git push and pull work from inside the sandbox without the agent ever handling the token itself."
A capability can still be misused while the VM is alive, and we wrote that down too: "the GitHub integration can publish repository content subject to GitHub's rules." We can't narrow it to one branch at the token level. GitHub installation tokens can be limited to specific repositories and permissions, and they expire after an hour, but there's no branch scope. Our PRD says it bluntly: "Do not describe the integration token as branch-scoped." The branch rule lives in two places instead: Atlas only ever sends one fixed push command to one predetermined atlas/* branch, and branch protection with required review guards main. The app has no bypass.
There's exactly one extractable secret in the guest: the model key. It's a dedicated OpenRouter key with a provider-enforced spend cap, it's revocable, and it's never our general key. If it leaks, the worst case is someone burns the rest of the cap. So "credential-free guest" in our design means something precise: no extractable secret whose blast radius is bigger than a bounded spend cap.
Agents are good at saying "all tests pass." Sometimes they're wrong. Sometimes they ran an earlier version. After a prompt injection, they might say it on purpose. Our PRD has one line for this: OpenCode's "statements about checks are never evidence."
Evidence is what the control plane observed. Atlas creates a candidate commit, then runs the checks of record against that exact SHA: typecheck, lint, tests, and task-specific checks against the seeded Postgres database. If they fail, the agent gets the output and tries again, up to three attempts. Each attempt makes a new candidate and reruns the whole suite.
If the third attempt still fails, Atlas can publish the result as a clearly failing draft marked CHECKS_FAILED, with every failing command visible and no claim of success. A failing draft with honest evidence still gives a reviewer something to debug.
Even a green candidate doesn't get pushed automatically. Atlas refuses to push when:
Some changes don't block a draft but get called out at the top of the PR: modified or deleted tests, .github/**, lockfiles and dependency changes, Prisma schema or migrations, auth and permissions, deployment config, and agent instruction files like AGENTS.md, CLAUDE.md, and .cursor/**. Those are the changes where a reviewer's attention matters most, and where an agent changing its own instructions would show up.
Then the push itself:
Recheck, immediately before pushing:
ticket still open
working tree clean
HEAD == candidate SHA
remote ref absent, or still at the expected prior SHA
hard publication policy still passes
Push:
1. Atlas sends a fixed push command over SSH
2. the VM pushes the candidate SHA to the predetermined atlas/* branch
3. Atlas asks GitHub for the remote head SHA
4. mismatch -> terminal failure, no PR openedStep 3 proves that what landed on GitHub is the exact commit Atlas checked. It doesn't prove that commit is good. That's what the checks, the callouts, and the reviewer are for. Atlas then opens the draft PR with the ticket link, base and candidate SHAs, the plan, check evidence, any failures, every callout, and an AI-generated disclosure. The same automated review that every human PR gets runs on it too.
In August, Atlas picked up a ticket to update body-parser. To do it, the agent regenerated bun.lock. One package in the tree was declared as "latest", unpinned. During regeneration it floated from 0.0.1 to a 1.0.0 that had been published about two days earlier, and dragged in roughly 190 new transitive packages.
Every check passed, and the PR went up green.
The only thing that caught it was Aikido Safe Chain on Evan's laptop. Safe Chain wraps package managers, blocks known malware, and by default refuses any version less than 48 hours old. It fired at the last possible moment, as "Evan can't install this branch."
Nothing we've seen suggests any of those packages was malicious. The problem was that our checks couldn't have told us either way. They test what the code does. They can't see where it came from or how long anyone has had to inspect it. The window just after a release is when a hijacked package does its damage. The Shai-Hulud worm compromised more than 500 npm packages in 2025 by stealing developers' tokens and publishing poisoned versions. pnpm's docs note that "in most cases, malicious releases are discovered and removed from the registry within an hour." An agent that resolves versions on its own needs the same delay the humans have.
The fix was policy parity: the same guard, running where the agent resolves versions, so the failure happens mid-run in front of the thing that can fix it.
The design choices, each from a measurement:
PATH export in the script that starts OpenCode, so every command the agent runs inherits it. Setup's bun install --frozen-lockfile stays off the proxy. A frozen install can't float versions, and in testing the proxy failed 2 of 6 cold installs.--safe-chain-skip-minimum-package-age, which the error message helpfully advertises.The best property turned out to be how Safe Chain enforces the gate. It removes too-new versions from the registry metadata the package manager sees. Ranges, bun update, and lockfile regeneration quietly pick the newest compliant version. Only an exact pin to a too-new version errors. The overhead was small: 11.3–11.6 seconds for a cold install of 2,192 packages, against 7.98 seconds without it.
Then we reran the original ticket. The agent noticed its own wholesale lockfile regeneration, reverted it, and landed an 18-line lockfile change instead. That PR was reviewed and merged. On a laptop running Safe Chain, the branch installed clean, and the package that first tripped the guard was confirmed still blockable and absent.
Package managers are adding the same idea natively. Bun has a minimum release age setting, in seconds. pnpm's equivalent, in minutes, has defaulted to one day since v11. We stayed with Safe Chain because it's what our engineers already run, and parity was the point.
Three other vendors document their approach publicly:
None of these designs relies on the model behaving. The OWASP Top 10 for LLM applications calls the failure mode excessive agency, and the fix in every design above is the same: enforce authority in code the model doesn't run.
apt, curl | bash installers, or Playwright's browser download. Closing this is the main reason we're re-testing Cloudflare's rebuilt sandboxes.Atlas may touch its workspace. It may not grant itself the authority to publish or merge. And because the network is open, it can still send out what it can read. Version one accepts that risk, in writing.
The same pattern of model proposing and deterministic code deciding runs through Kai's deal agent, the MCP tool surface, and the approval ladder in our product. For how we handle customer data, see Security.
We'll be at Cloudflare Connect October 19–21. If you're going to "Your agent wrote it. Who's on call?" or "Securing agentic harnesses" on Wednesday, come find us. We'd like to compare threat models.
Introzy is free to start. No card required.