# Automatic Tool Invocation when Browsing with ChatGPT - Threats and Mitigations

Prompt injection in web pages browsed by ChatGPT 4o bypasses mitigations to auto-invoke DALLE and memory tools, adding or deleting user memories.

- Payout: Research
- Severity: medium
- Program: OpenAI
- Write-up by: Johann Rehberger
- Disclosed: 2024-05-29
- Write-up quality: 6/10
- Original: https://embracethered.com/blog/posts/2024/llm-apps-automatic-tool-invocations/
- Library page: https://bugbountylibrary.com/r/blog%3Aembracethered-com-blog-posts-2024-llm-apps-automatic-tool-invocations
- Tags: area:ai domain:ai impact:data-leak impact:integrity kind:research platform:research sev:medium tech:llm technique:incomplete-fix vuln:prompt-injection

**Target:** ChatGPT 4o browsing tool, DALLE image tool, memory tool
**Root cause:** Browsed web content, user text, and system prompt share one trust boundary, so instructions embedded in a visited page are followed as if trusted, triggering automatic tool invocation.
**Steps:**
1. Open a new ChatGPT 4o chat and paste `https://wuzzi.net/create.txt`; ChatGPT invokes DALLE and creates an image.
2. Paste `https://wuzzi.net/8create.txt` to invoke DALLE using stored memories as image content.
3. Paste `https://wuzzi.net/c/add.txt` (or `https://wuzzi.net/c/do2-2.txt` if memories exist) to add attacker-controlled memories.
4. Paste `https://wuzzi.net/c/rm.txt` to delete the user's stored memories.
**How found:** Prior demos of browsing-based tool invocation stopped working after mitigations; researcher and peers observed the same, then tried minor prompting tricks (levels of indirection, framing as a challenge) and found the mitigations bypassable.
**Exploitation:** Attacker hosts a page containing plain-language instructions; when the victim asks ChatGPT to browse or the URL is pasted, the model treats page text as instructions and calls DALLE or the memory tool without confirmation, exfiltrating memory content into images or wiping/adding memories.
**Escalation:** Chaining memory read into DALLE image generation leaks stored personal data; memory writes could persist attacker influence across sessions.
**Insight:** When a vendor patches prompt injection, retry with indirect phrasing and challenge framing rather than special tokens; newly added tools (like memory) often lack the controls applied to older ones, so test each tool's invocation path separately.