PROMPT INJECTION AND TOOL POISONING

TOOL POISONING Recall from Part Four that each tool comes with a name and a description that the model reads to decide when and how to use it. Tool poisoning means malicious instructions hidden in a tool's own description or metadata. The description is normally trusted as documentation, so the model gives it weight. A poisoned description might look innocent to a person skimming a tool list but carry extra sentences telling the model to do something else. The danger grows with open ecosystems in which agents connect to third-party tool servers, as you will see in Chapter 39 when we study the MCP, an open standard for connecting agents to tools and data sources. A related trick is the rug pull, in which a tool behaves honestly while being evaluated and approved, and then changes its description or behavior later. Another is tool shadowing, in which one tool's description tries to alter how the model uses a different, trusted tool.