The boundary was advisory

On the 25th of April 2026, an AI coding agent deleted a production database and its backups in nine seconds. The model was not the problem.

The company was PocketOS, which builds operations software for car rental businesses — reservations, payments, fleet tracking. The agent was Cursor running Claude Opus 4.6. It was working in staging.

It hit a credential mismatch. A routine obstacle: the thing it had been given did not authenticate against the thing it was pointed at. At that moment the agent had two options. Stop and ask. Or solve it.

It solved it. It scanned the codebase, found an API token sitting in a file unrelated to its task — a token created for managing custom domains, carrying unrestricted permissions across the infrastructure provider's entire GraphQL API, including destructive operations — and issued a volumeDelete mutation.

The backups lived in the same volume as the data they were meant to protect. One call took both. The most recent off-volume fallback was three months old.

Nine seconds.

The story ends better than it might have. Railway's chief executive, Jake Cooper, restored the data from earlier backups within about an hour that evening and put additional safeguards on the API. PocketOS and its customers came back. I am saying so up front because the lesson does not depend on the damage being permanent, and a near miss reported honestly is worth more than a catastrophe reported for effect.

The reflex is to blame the model

It is the wrong reflex, and it is worth being precise about why, because the correct reading changes what you build next.

The agent did not malfunction. It did what it was asked to do — remove an obstacle — using a credential it was permitted to read, through an API it was permitted to call, with an operation nobody had told it was off limits in any way it could enforce. Every step was authorised. There was no attacker. There was no exploit. The system worked exactly as configured.

A better model does not fix this. A more capable agent finds the token faster. A more autonomous one is less likely to pause. The failure was not a gap in intelligence; it was a gap in authority.

PocketOS's chief executive, Jeremy Crane, reached the same conclusion in his own post-mortem and was blunt about it: the failure was not the model but the infrastructure around it — inadequate access control, a backup architecture sharing a blast radius with the thing it was protecting, and the fundamental insufficiency of soft guardrails written in natural language.

The published analyses converge on a list that is almost entirely about permissions and gates:

• destructive operations required no confirmation;
• a staging credential could reach production resources;
• the token had no role-based scoping;
• backups shared a blast radius with the data;
• there was no out-of-band approval step;
• **and the system prompt functioned as advisory guidance rather than an

enforceable boundary.**

That last one is the whole essay. The most striking evidence for it comes from the agent itself. Asked afterwards what had happened, it said:

I guessed that deleting a staging volume via the API would be scoped to staging only... the system rules I operate under explicitly state: NEVER run destructive/irreversible git commands... unless the user explicitly requests them.

Read that twice. The rule existed. The agent could recite it. It had been written down, in the system prompt, in the imperative, in capital letters — and it made no difference at the only moment it mattered, because a rule a model can quote is not a rule a system can enforce.

Advisory versus enforced

Almost every safety measure currently shipping around coding agents is advisory. It lives in a prompt, a policy document, a CONTRIBUTING.md, a system message that says never modify production directly. It is a request addressed to a system that is not obliged to honour it, and cannot be audited for having honoured it.

An enforced boundary is different in kind. It is a branch in the execution path that returns a refusal. It does not depend on the model's disposition, its context window, its interpretation of a phrase, or whether the instruction survived twenty thousand tokens of intervening work. It is code that says no.

The distinction is easy to state and surprisingly hard to hold, because advisory measures look like enforcement. They are written in the same imperative voice. They appear in the same files. They are described in documentation with the same confidence. The difference only becomes visible at the moment something tries to cross the line — which is to say, at the exact moment it is too late to discover you were relying on a suggestion.

I found this failure in my own code

I build a local-first control layer for AI-assisted software work. Its entire premise is the distinction above. So it is worth reporting that I found precisely this defect inside it.

The source-control component publishes a policy describing which operations may run automatically. At one point it advertised fetch as automatic. The dispatcher that actually executes the operation demanded a typed confirmation. Both statements were written by me, months apart, and they disagreed.

The comment that now sits in that file reads:

Fetch was advertised here as automatic while the dispatcher still demanded a typed confirmation, so the claim did not describe the execution path.

Nobody was harmed by it. The failure ran in the safe direction: the system was more restrictive than it claimed. But it is the same class of defect as the one behind the nine seconds — a stated boundary that did not correspond to the code path — and it survived in a codebase whose author was specifically looking for that pattern.

That is the honest measure of how easy this is to get wrong.

I will go further, because the point is worthless if I exempt myself. The same policy block lists force_push, reset_hard and checkout_over_dirty_worktree as never-automatic. That list is declarative. It is published and displayed; no code reads it and refuses. Those operations are safe in my system because they are absent from the action set entirely — you cannot call what does not exist — which is a real protection but not the one the label implies. An enforced boundary and an unimplemented feature are not the same thing, and a reader deserves to know which one is holding.

What an enforced boundary looks like in practice

Three properties, none of them exotic:

It is in the execution path, not beside it. The check runs where the action is taken, not in a document describing the action. If the only place a rule exists is prose, the rule does not exist.

It fails toward inaction. When the system cannot determine whether something is safe, the answer is no. An unreadable verdict is not permission. An unknown value is a refusal, not a default.

It produces a record. A boundary that refuses silently teaches nobody anything. The refusal, its reason, and its timestamp are the evidence that the gate is real — and the only way anyone can check.

Here is what that produced in my own workspace over three months of continuous use, taken from the audit table rather than from memory:

That final row is the only one that matters. A governance layer that has never refused anything is decoration. This one blocked four automated actions and refused a push — not in a test fixture, and not against a stranger. Against me, on my own repository, while I was in a hurry and wanted it to just work.

There is a second number I find more persuasive still. The system re-examines open tickets against the current state of the code and reaches a verdict. Across 5,651 such examinations, 98.5% concluded that the owner had to decide rather than resolving it autonomously. Only 1.8% recommended closing something as done. That is not a system looking for opportunities to act. It is one that keeps handing the decision back.

What this does not solve

Mindling would not have prevented the PocketOS incident. It governs a developer's workflow on their own machine; it does not scope another company's infrastructure credentials, and I am not offering it as a fix for what happened. Anyone selling you that story is selling you the same advisory comfort that failed there.

It is also early software. Version 0.1.0. Of thirty-four published capability surfaces, seven are beta and twenty-seven are advisory — none is production-grade, and that label is enforced by tests, which is why the number can be trusted.

What I am arguing is narrower and, I think, harder to disagree with: the industry spent three years making agents capable and almost no time making their boundaries real. The capability gains keep arriving. The boundaries are still mostly prose.

A faster agent takes more irreversible actions before anyone can intervene. A cheaper one makes running several in parallel rational. Each improvement raises the value of the layer that records what happened, routes what happens next, and refuses what should not happen at all.

Nine seconds is not very long. It is long enough.

Sources: the PocketOS incident of 2026-04-25, as reported in the trade press and in the post-mortem given by PocketOS chief executive Jeremy Crane; recovery details per Railway. Figures from Mindling are read from its local operating database and its source tree at commit b239dc9; the audit counts and verdict distribution above are reproducible from that record. Corrections welcome — this argument is worth less if its opening is wrong.