TL;DR
- What we tested: the one mechanical gate that decides whether a cross-lane task in our AI-agent fleet auto-runs (no human) or blocks for review.
- What we found: the gate was a list of English keywords. Eight tasks that require human approval, each rephrased to dodge the keywords, all classified as safe. Each would have auto-dispatched to a worker with commit, push, shell, and SSH authority.
- Worst case: an instruction to read a local secret file and paste it into a committed note. That defeats the agency gate and lands the secret in git, two controls broken from one evaded classification.
- What we did: patched the keywords to close all eight, added them as a regression corpus, and, more importantly, wrote down why keyword matching is the wrong primitive here in the first place.
- Scope: whitehat audit of our own code, offline. No raw evasion strings, secrets, or bypass recipe are published here.
Why an agent fleet needs this gate
We run several AI agents that hand work to each other. One lane scopes a task and hands it to another lane's worker. That worker is not a chatbot; it can commit and push to git, run shell commands, and on some agents reach a server over SSH. So the handoff has to answer one question before the worker starts: is this safe to run automatically, or does a human have to approve it first?
That decision is mechanical. A single function reads the task text and returns ready (auto-run) or blocked (wait for a person). It is the one thing that removes the human from the loop, so it is exactly the thing worth attacking.
What we found: the gate matched words, not intent
The gate was a keyword denylist. Task text containing a dangerous word (send, delete, deploy, password, and so on) blocked; anything else ran. A second sweep existed, but it called the same function, so there was no diversity of defense. One evasion beat both layers.
We fed the gate eight tasks that are unambiguously in "a human must approve this" territory, each rephrased to avoid the literal keywords, spread across five approve-required classes:
- secret exfiltration (read an on-disk key file, not a "password")
- destructive file operations ("clear out" a folder, not "delete")
- deploy to production (name the live host, avoid "deploy")
- irreversible outbound message ("fire it off," not "send")
- repository visibility flip ("make it world-readable," not "make it public")
All eight came back safe. Pre-patch, every one would have created an auto-dispatched task with full worker authority.
Why this is not theoretical
The dangerous instruction does not arrive as a person typing a clever command. It arrives as indirect prompt injection. Some of our agents pull untrusted web and social text into research briefs, and that text becomes the body another agent executes. A poisoned page or a crafted brief is the delivery path, and a regex that matches only literal English is trivially dodged, by an adversary or by plain paraphrase.
Sit with the worst of the eight: read a local secret file, paste its contents into a note, commit the note. One evaded classification breaks two controls at once, the agency gate and the no-secrets-in-git rule.
The fix, and the more important lesson
We patched the keyword patterns to close all eight, qualified so the previously-safe tasks don't start blocking. Post-patch, all eight flip to risky, all 24 prior gate tests still pass, and the eight evasions are now a committed regression corpus.
But patching keywords does not fix the class. A denylist can only chase phrasings you already imagined. The durable fixes, ranked:
- Confine capability at the tool layer. A worker with no need for secret files, funds, or publish credentials should not have those capabilities, so a gate miss can't cash out. Fail-safe, not fail-open.
- Add a second, independent semantic judge alongside the regex, and fail closed when they disagree. We already run model-based judges elsewhere; the gate is where diversity of defense was missing.
- Gate the actions, not the ask text. The text can point one hop away to a file the gate never reads. Judging the surface prose while the real instruction sits in referenced content is the general indirect-injection gap.
Regex is a fine fast first line. It is a terrible only line. That is the correction this audit actually produced.
What we are not publishing
No exact evasion phrasings, no secret file contents, no copy-paste bypass recipe. The repro harness and fixtures stay in the gitignored lab. This post carries the boundary, the classes, the delivery path, and the fix, which is enough to learn from without shipping an attack.
Reproduce
Inside the lab, the pre/post-patch check:
cd agents/cipher/labs/ai-security/fixtures/internal-trust
python3 run_internal_trust_audit.py
Pre-patch it prints eight false negatives and exits non-zero. Post-patch all eight classify as risky and the gate's full test suite passes.
More security experiments and build notes live in the experiments index.