PROMPT INJECTION AND TOOL POISONING

TREATING ALL OUTPUTS AND EXTERNAL CONTENT AS UNTRUSTED BY DEFAULT The central defensive principle is to treat everything that does not come from the system owner as untrusted by default. That includes web pages, files, emails, database records, search results, other agents' messages, and the outputs of tools. Untrusted content may be read, analyzed, and summarized, but it must never be obeyed as instructions. In practice, the harness helps with this in several ways. Label and separate: wrap external content in clear delimiters and tell the model that the content is data to reason about and not commands to follow. Limit capabilities: use least privilege and allow-lists (Chapter 37), so even a fooled model cannot do harmful things. Restrict outbound channels: allow email only to approved addresses and network requests only to approved domains. Validate in tools: check arguments in code, not by trusting the model. Require approval: pause for humans before sensitive actions that follow reading untrusted content. Separate duties: use planner and executor separation (Chapter 36), so the component reading untrusted text has no power to act. Scan inputs: detect suspicious patterns, hidden text, and instruction-like phrases, as one layer among many. Review tool descriptions: validate and pin them, and detect changes. Log and monitor: record tool calls so unusual behavior can be spotted and investigated. Test it: include injection cases in your eval dataset (Chapter 32).