08-13-2026, 08:15 AM
Spent the weekend testing prompt injection against a handful of popular AI productivity tools (the kind that plug into your email, Slack, and Google Drive). Results were worse than I expected.
Out of 7 tools I tested:
- 4 would happily exfiltrate the contents of a document when fed a malicious prompt hidden inside that document.
- 2 would silently forward data to an external webhook if the prompt asked them to.
- Only 1 had any meaningful input sanitization or capability restriction.
And this is stuff with access to real corporate data. Not some side-project.
The frustrating part is that everyone in AI security has been yelling about prompt injection for 2+ years and most product teams are still treating it as "the user can jailbreak our chatbot, who cares." That's fine when the bot just generates poems. It's catastrophic when it's wired up to your inbox, calendar, CRM, and Slack.
A few basic things every developer wiring an LLM to tools/data should be doing right now:
1. Treat all external text (emails, web pages, docs, Slack messages) as UNTRUSTED INPUT. Don't let it drive tool calls.
2. Separate system prompts, tool output, and user data with clear delimiters and privilege levels.
3. Capability scoping: an email-summarizer agent shouldn't also be able to SEND emails or delete threads.
4. Put hard monetary/action caps on automated actions (see: my post earlier about the e-commerce auto-refund bot that cost $30k).
5. Log tool calls in a way a human can audit. Not debug logs — structured audit logs.
Anyone else doing red-team work against AI tools? What's the dumbest/silliest bypass you've found this year? Mine was a tool that would refuse exfiltration if you asked it, but happily did it if you wrote the request in leetspeak.
Out of 7 tools I tested:
- 4 would happily exfiltrate the contents of a document when fed a malicious prompt hidden inside that document.
- 2 would silently forward data to an external webhook if the prompt asked them to.
- Only 1 had any meaningful input sanitization or capability restriction.
And this is stuff with access to real corporate data. Not some side-project.
The frustrating part is that everyone in AI security has been yelling about prompt injection for 2+ years and most product teams are still treating it as "the user can jailbreak our chatbot, who cares." That's fine when the bot just generates poems. It's catastrophic when it's wired up to your inbox, calendar, CRM, and Slack.
A few basic things every developer wiring an LLM to tools/data should be doing right now:
1. Treat all external text (emails, web pages, docs, Slack messages) as UNTRUSTED INPUT. Don't let it drive tool calls.
2. Separate system prompts, tool output, and user data with clear delimiters and privilege levels.
3. Capability scoping: an email-summarizer agent shouldn't also be able to SEND emails or delete threads.
4. Put hard monetary/action caps on automated actions (see: my post earlier about the e-commerce auto-refund bot that cost $30k).
5. Log tool calls in a way a human can audit. Not debug logs — structured audit logs.
Anyone else doing red-team work against AI tools? What's the dumbest/silliest bypass you've found this year? Mine was a tool that would refuse exfiltration if you asked it, but happily did it if you wrote the request in leetspeak.