Prompt injection in a pipeline

A factory’s agents spend their time reading text written by people outside the team. Tickets filed by requesters, comments on changes, log lines carrying user input. A language model cannot reliably tell an instruction from a description of one, which means every one of those is a place where somebody can try to give your automation an order.

The attack is not hypothetical#

In May 2025 Invariant Labs demonstrated the shape of it. A malicious issue filed on a public repository, an assistant asked by its own operator to go and look at the open issues, and instructions hidden in the issue body steering that assistant into reading private repositories and publishing what it found. The developer did nothing unusual. They asked a reasonable question about their own project.

The part worth sitting with is the researchers’ conclusion: this was not a defect in the server’s code, and there was no obvious fix. The assistant had legitimate access to private repositories, it had been pointed at content an attacker controlled, and it could publish. Every individual permission was intentional. The combination was the vulnerability, and no amount of careful implementation of any single piece would have removed it.

The mitigations the researchers landed on are telling, because none of them is about the model: one repository per session, and least-privilege tokens. Both are answers about what the run is allowed to reach, not about how it is asked to behave.

Remove one leg per agent#

Simon Willison’s framing is the one to build against, because it turns a vague worry into a checkable property. Three capabilities: access to private data, exposure to untrusted content, and the ability to communicate outwards. Any two of them together are harmless. All three in one run is an exfiltration path regardless of what the instructions say.

The design rule that follows is short enough to apply per agent, which is what makes it useful: give each one two legs and never the third. And this is where the narrowness of the agents stops being an aesthetic preference and starts paying for itself, because a single broadly capable agent has all three by construction and there is nothing left to take away.

Concretely. A readiness agent reads ticket text an outsider wrote and can reach the repository — so it holds no write credentials for code at all, and cannot publish anything beyond a comment on the item it was given. An agent that triages production errors reads log lines that may contain attacker-controlled strings and has access to real system internals — so it has no network route out except to the tracker. A builder holds broad write access to a branch and reads the codebase — so its input is the structured work item and the code, never free-form text from outside the team.

Note what that last one costs. The builder cannot read the original ticket description, only the validated hand-off derived from it. That is a real constraint on how much context reaches the implementation, and it is the price of the leg being removed. Design rules that cost nothing usually are not doing anything.

What is left over#

Model-level defences are real and improving. Anthropic reports adaptive attacks against Claude Opus 4.5 in browser use succeeding around 1% of the time, down substantially from earlier versions — screening content before the model acts on it, classifiers over tool results, training. That is genuine progress and it is also a rate.

Which is the whole argument in one sentence: at the volume a factory runs at, a 1% success rate against a determined attacker is not a safety margin, it is a schedule. Anything that must not happen has to be impossible rather than unlikely, and the only way to make something impossible is to remove the capability rather than to discourage its use.

So the controls that matter are the boring ones, and they sit where no model is involved. Short-lived credentials scoped to one repository and one project, injected at run time. An outbound network allowlist per agent. Hard rules enforced by hooks that can block an action before it executes, rather than by sentences in an instruction file. Every external write going through the same path — the model produces a structured proposal, a schema validates it, a policy checks it against what this workflow is permitted to do, and ordinary code executes it. An instruction guides a model; a hook stops it.

And behind all of that, one control that does not depend on getting any of it right: nothing merges without a person reading it. Prompt injection is the clearest reason that gate is not a temporary measure. A defence with a residual success rate needs something behind it that does not, and a person reading a diff is the only layer in the stack that an instruction hidden in a log line cannot address. The question to ask of every agent you add, before anything else, is which of the three legs it does without — the same question the hand-off design has to answer about what each agent is allowed to reach.

Bounding what an agent may do limits the damage. Knowing whether it still works after you have bounded it is a different question, and the first answer is not a better prompt but a smaller blast radius — what the agent is allowed to reach, and how you stop it.