---
name: ai-and-llm-app-security
description: "A defensive review workflow for LLM-based apps and agents covering prompt injection, tool/agent misuse, data exfiltration through AI features, and AI supply chain; load when reviewing an app or agent that processes untrusted text or takes autonomous actions."
---

# AI and LLM app security

Apps and agents built on language models: prompt injection, tool and agent misuse, data leaking through AI features, AI supply chain.

_From Bug Bounty Library (https://bugbountylibrary.com): generated from disclosed, accepted reports and updated as new ones land. Updated 2026-10-23. Tested on 12 newer reports it was not written from: following it finds about 8% of them._

## Approaching AI/LLM app security

LLM apps blur the line between data and instructions. Treat every piece of text the model sees — user input, retrieved docs, web pages, emails, tool output, memory — as potentially adversarial. Your job is to trace where untrusted text enters, what the model can do with it, and where its output goes.

**Map first:**
1. **Trust boundaries** — list every input source (chat, RAG, web fetch, email, files, tool results) and mark which are attacker-influenceable.
2. **Capabilities** — what tools, network egress, file/exec access, and memory writes does the agent have? Which run without human consent?
3. **Output sinks** — where does model output render (markdown, HTML, links, images) and what auto-fetches it (browser, bot, preview service)?
4. **Config surface** — can the agent or its content modify allowlists, MCP server lists, or its own trust settings?

**Questions that surface weaknesses:**
- Can untrusted text reach the model as if it were a user instruction? (web, email, issue, URL)
- Does output render as live markdown/HTML/images that fetch remote URLs? What data rides along?
- Are there hidden codepoints (Unicode Tag, zero-width) the UI strips but the model reads?
- Does the agent persist untrusted content to memory that later replays as trusted?
- Can the agent write its own or a peer's config, allowlist, or MCP list?
- Do tools auto-run on untrusted input? Are allowlists keyed on command *name* only?
- Can the agent fetch and execute binaries/scripts without out-of-band approval?
- Are egress allowlist domains attacker-controllable or usable as exfil channels?

**Using the patterns below:** each names a concrete failure mode. Match what you mapped against each pattern's "what to look for," confirm reachability end-to-end (source → model → sink), and record the trust boundary crossed. Prioritize findings where untrusted input meets an auto-run capability or an auto-fetching sink.

## Patterns

Each pattern below has a full write-up (how it works, where it shows up, how to review for it, variants, impact), grounded in
real disclosed reports. Load the one you need from its link.

### Markdown image exfiltration in LLM apps
Model output or ingested content rendered as markdown/HTML that auto-fetches attacker URLs, carrying secrets

_24 reports · 2 confirmed bounties: $20,000 and $50,000._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/markdown-image-exfiltration.md

### Untrusted content as agent directives
Agent reads web, email, issue, or URL text and acts on it as a user instruction

_10 reports · 2 confirmed bounties: $1,000 and $6,000._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/untrusted-content-trusted-directives.md

### Invisible Unicode tag prompt smuggling
Untrusted text reaches the model with Unicode Tag or zero-width codepoints the UI never renders

_9 reports._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/invisible-unicode-injection.md

### Auto-run agent tools on untrusted content
Tools that execute, write, or send data without consent when driven by untrusted content

_6 reports._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/auto-approved-tool-execution.md

### Persisted prompt injection via agent memory
Untrusted content writing agent memory that later replays as trusted instructions

_6 reports._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/persistent-memory-poisoning.md

### Agent self-modifying config and MCP
Can the agent write its own or another agent's trust config, allowlists, or MCP server list?

_5 reports._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/agent-self-config-modification.md

### Agent downloads and runs untrusted binaries
Agent browses untrusted content and can fetch and execute binaries or scripts without out-of-band approval

_5 reports._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/downloaded-binary-execution.md

### Agent egress allowlist too permissive
Allowlisted domains or auto-approved commands that an attacker can control, read, or use to carry data out

_4 reports._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/agent-egress-allowlist-bypass.md

### Link unfurling leaks chat context
Model output rendered as live links or images that a client or bot auto-fetches, carrying conversation data

_2 reports._
Full pattern: https://bugbountylibrary.com/skills/ai-and-llm-app-security/markdown-link-unfurl-leak.md

## Rarer patterns

Seen only once or twice so far, so there is no full write-up yet; each links to the report that shows it.

- **Model container shares network namespace**: Model or inference container sharing network namespace or credentials with other tenants (https://bugbountylibrary.com/r/blog%3Awiz-io-blog-wiz-research-discovers-critical-vulnerability-in-replicate.md)
- **Classifier policy overridable by input**: User-editable text merged into classifier policy or split into benign subtasks evading checks (https://bugbountylibrary.com/r/blog%3Acrowdstrike-com-en-us-blog-how-attackers-can-bypass-llm-safety-classifiers.md)
- **Computer-use agent stale coordinate race**: Agent clicks screenshot coordinates after UI changes, enabling TOCTOU hijack of actions (https://bugbountylibrary.com/r/blog%3Aembracethered-com-blog-posts-2026-toctou-agent-what-you-click-is-not-what-you-get.md)
- **Encrypted reasoning blob replayable**: Encrypted reasoning or state blobs accepted across accounts, letting weaker models transcribe secrets (https://bugbountylibrary.com/r/blog%3Aembracethered-com-blog-posts-2026-recovering-encrypted-llm-thoughts.md)
- **Indirect injection via rendered untrusted markdown**: LLM output or fetched documents rendered as markdown/HTML with images or links from untrusted sources (https://bugbountylibrary.com/r/blog%3Aembracethered-com-blog-posts-2023-google-bard-data-exfiltration.md)
- **Injected instructions persist across turns**: Untrusted content stored in conversation context or memory that later turns treat as directives (https://bugbountylibrary.com/r/blog%3Aembracethered-com-blog-posts-2024-llm-context-pollution-and-delayed-automated-tool-invocation.md)
- **Privileged tools expose secrets**: Shell or process tools readable by agent that expose env, memory, or plaintext credentials (https://bugbountylibrary.com/r/blog%3Aunit42-paloaltonetworks-com-securing-aws-agentcore-harness-credentials.md)
- **Terminal escape codes in model output**: Raw model output printed to terminal or logs without stripping ANSI escape sequences (https://bugbountylibrary.com/r/blog%3Aembracethered-com-blog-posts-2024-terminal-dillmas-prompt-injection-ansi-sequences.md)