Scoped credentials and the kill switch

You have written down what the agents must not do. It is in the instruction files, it is in the prompt, and every run reads it. Then one run does it anyway — because a language model treats an instruction as strong advice, and a factory runs enough times a day that strong advice is not a control. The question is what remains true when the text is ignored.

A prompt is a request, a permission is an answer#

Anything an agent must never do should be something it cannot do. That single rule converts most of what teams write as guidance into something the platform decides before the model sees a token. A prompt asks. A permission answers, and it answers the same way whether the run is behaving well, confused, or being steered by somebody else.

The first place it applies is the credential. Agents are usually given a service account with the breadth of a helpful colleague — every repository, every project, indefinitely valid — because that account was created once and reused. The alternative is per run: a short-lived token scoped to exactly what this run needs, one repository and one project, minted from a secrets store when the run starts and worthless shortly after it ends. It is never committed, never baked into an image, and never covers production by default. If a run genuinely needs production, that is a named exception with an expiry on it, not a property of the account.

Next comes the matrix, and the word that carries it is per workflow. Permissions written per system produce one answer for the tracker and one for the repository, which is another way of saying every agent gets the union of what any agent ever needed. Written per workflow, each row states separately what that kind of run may read and what it may write across the tracker, the repository, branches, review, deployment and production. Most of the interest is in the cells you leave empty.

Dwell on the first row for a moment, because it looks like an inconvenience and is not. A planning agent that may comment on a ticket but has no write access to its description field cannot damage the requester’s own words — not through a bug in its instructions, and not through an instruction hidden inside the very ticket it is reading. The requester’s text stays the record of what was asked, and every later step is checked against it. A convention would have achieved that on most days. The empty cell achieves it on all of them.

Access is only half of it, though, and the half that gets attention. What a run can reach outwards decides whether anything it learns can leave, so each agent also gets an outbound network allowlist: the tracker, the forge, the model endpoint, nothing else. An agent that reads log lines an outsider may have written, holds credentials to real systems, and can also reach the open internet has the full set that makes exfiltration possible: private data, untrusted text, and a way out. Take the third away and the other two are harmless together. Restricting egress is unglamorous, and it is enforced by infrastructure rather than judgement — which is what makes it the leg that stays removed.

Four levels of stop, not one#

Now the other half of the same idea. Permissions decide what a healthy system allows; a stop decides what happens when the system is not healthy, and it needs at least four settings. A global stop that disables every agent run there is. A workflow stop that disables one kind of agent while the others keep going — turn the builder off, leave the advisory reviewer running. A service stop that disables agents on one part of the system only. And run cancellation, which terminates one active run and revokes what it was holding.

In practice the second and third settings are the ones incidents call for, and they are the two that get built last. A team with only the master switch faces a bad choice every time something looks wrong: stop everything and lose the day, or leave it running while somebody works out how serious this is. The predictable outcome is that nobody pulls the switch, and the argument about severity happens while the thing continues.

Two properties decide whether a stop is real. The first is where it is enforced. A label on a work item, an assignee change or a paused schedule can each be quietly undone by the next automated sweep — and a factory is full of automated sweeps, since that is what you built it to be. The stop has to live in the platform that decides whether a run starts at all, checked at dispatch, with no path around it.

The second is the credential. Stopping the dispatcher prevents new runs; it does nothing about the twelve already executing, each holding a valid token in a container that is still running. Every credential issued to an active run therefore has to be revocable on demand, or short-lived enough that it expires within minutes regardless. A stop that leaves working credentials in flight has stopped the future and left the present alone, and the present is the part causing the incident.

Then there is the part no diagram covers. A kill switch nobody is named for is not a control. Somebody has to be able to use it at three in the morning without asking permission, from a phone, without a deployment — and choosing that person during the incident costs the minutes the switch existed to save. Two or three names, written where the on-call rota is written, and exercised the way a change to the shared instruction set is: by actually running it.

The permissions nobody withdrew#

Almost every real failure here is a slow one. A run kept failing on something nobody could reproduce, somebody widened the token for an afternoon to see what was happening, and it worked. The agent has held that token for seven months. Nobody granted it permanently; nobody withdrew it either, because withdrawal was never anybody’s task and the system gives no signal that anything is wrong. The fix is structural rather than cultural: every widening carries an expiry at the moment it is granted, so the default direction is back to narrow and a second person has to decide the reason still holds.

The stop rots differently. It was tested once at rollout, it worked, and it has not been exercised since — while the orchestrator was rewritten, the queue moved, credentials went from static to minted, and a new workflow was added by a team that did not know the flag existed. The next time anybody presses it will be the first time in a year, during an incident, with no idea what it actually covers. Pull it on a quiet Tuesday, on a schedule, and watch what keeps running. Anything still going is a gap you would otherwise have found at the worst possible moment.

The third failure is the one that erodes the whole design, and it never announces itself. A narrow token made a run fail on a Friday afternoon. Somebody replaced it with a broader one to unblock the release, meaning to tighten it on Monday. Repeat that a dozen times over a year and the matrix on the wiki no longer describes the system — every agent holds roughly what every other agent holds, and the empty cells that were the entire point have quietly filled in. The counter-pressure has to be as ordinary as the pressure: a permission change is a reviewed change, the matrix is generated from the configuration rather than maintained beside it, and a run that fails on a narrow token is a bug in the scope, not an argument for widening it.

None of this is expensive relative to the rest of a factory. Short-lived tokens, an egress allowlist, a stop with four settings and two named people who may pull it come to about a week of work, and they decide whether the worst case is bounded by what a run could reach or by whether every run behaved. The same instinct gives each run its own disposable environment and its own data.

Bounds tell you the worst case. They say nothing about whether the thing still works inside them, and that is a question you can only answer by running the same inputs again and comparing — a golden set.