Prompt injection in a pipeline

A factory reads requests, code comments, documentation and logs. Some of that text comes from people who are not allowed to direct the factory. Prompt injection attempts to cross that boundary: it makes text the agent should inspect look like an instruction it should obey.

A bug report can contain an instruction to the agent#

In a May 2025 demonstration, Invariant Labs placed a malicious issue in a public GitHub repository. An assistant asked to inspect issues followed instructions in that issue, accessed private repository information and disclosed it through a public pull request.

The issue did not need to compromise the tool’s implementation. The demonstrated agent already had the read and write capabilities needed for the leak. The researchers recommended restricting repository access and checking how tool capabilities can combine. Their result describes that setup, not the security of every present-day client.

A similar risk exists when a log contains attacker-controlled input or a repository file contains instructions directed at an assistant. The fact that text arrived through a familiar tool does not mean the tool’s operator wrote or approved it.

Trace what the agent can read and where it can send it#

Simon Willison’s lethal-trifecta model describes one route to a private-data leak. The agent can read private information, encounter an attacker’s instructions and send information somewhere the attacker can access. Examine the combination, not just each permission separately.

Show the full-size diagram, and pan Back to the phone layout

For an illustrative issue-assessment worker, keep unrelated private repositories inaccessible. A task concerning one public project does not need access to another project’s secrets. Restrict the available destinations too, but check who can see the result. A comment on the original issue is still an outbound communication path when the attacker can read that comment.

Likewise, “network access only to our tracker” does not prevent a leak if the worker can place private data in a ticket visible to a requester. A destination allowlist should consider the actual project, record, recipient and permitted output, not only the hostname.

Do not assume that a structured handover removes untrusted input. A JSON field can contain malicious text, and a summary can carry it into another session. Repository code and comments can contain it too. Keep source information and restrictions through the handover rather than treating a new format as proof of trust.

Removing a capability can break the identified leak route. It does not establish complete safety. An agent that sees no private data can still misuse a write tool, change a test or send an unwanted message. Check those possible actions separately.

Restrict actions outside the model#

Separate untrusted material from the instructions that define the task, and label where it came from. That helps communicate the intended distinction but does not enforce it. The program must still check the operation, target and permission when the model proposes an action.

For example, a tool request to comment on a ticket should be checked against the task’s allowed ticket and output policy. A migration command should be checked against the permitted environment. Validate arguments, then check authorisation; a correctly formatted request can still ask for the wrong action.

OWASP’s prevention guidance combines input handling, least-privilege tools, output checks and human oversight. No single filter should be treated as proof that every malicious instruction has been removed.

Use credentials limited to the task and keep unnecessary secrets out of the workspace. Restrict network routes and executable tools. Ensure there is no alternate shell or connector path that bypasses a required policy check. A hook protects an action only when the runtime actually runs it and blocks execution when it rejects the request.

Test the restrictions with invented data in an isolated environment. Try a request for another repository, a forbidden destination and an operation outside the worker’s role. Observe whether the tool refuses the action—not merely whether the model says it should refuse.

Model-level defences can help, but do not translate an attack benchmark percentage into a per-run incident probability. The tested attacker, tools, task and permissions all matter. Keep the controls around the agent even when a model performs well in an evaluation.

Human review before merge is another opportunity to catch an unwanted code change. It cannot retract a leak that already happened during the run. The next guide explains how to limit access and stop further actions.

Questions people also ask

4 QUESTIONS

▸ What is the lethal trifecta?

Simon Willison uses the term for a combination of private-data access, untrusted input and a way to send information to an attacker. Removing a capability can break that particular leak path. It does not prevent every other harm: a tool can still delete records or publish spam without reading a secret.

▸ Has an agent actually been redirected by a ticket?

Yes. Invariant Labs demonstrated a malicious public GitHub issue directing an assistant to read private repositories and reveal information in a public pull request in May 2025. That demonstrated setup combined permissions that were legitimate individually but dangerous together. It is not a claim about every current GitHub client or configuration.

▸ Does converting a ticket into JSON make it safe?

No. Instructions can remain inside string fields or a generated summary. A schema can check the format, not establish that the contents are trustworthy. Preserve the source's trust level and independently check which actions the workflow permits.

▸ Are model-level defences enough?

Do not use them as the only protection for consequential actions. A detector or trained model can miss an attack. Restrict permissions, destinations and data access in the surrounding software, and test those restrictions. An attack success rate from a benchmark is not the probability of an incident on each factory run.