CONTEXT ROT AND CONTEXT BLOAT

CONTEXT BLOAT Let us start with the simpler problem. Context bloat means the window fills with too much low-value text. The word bloat is used the way you might say a person feels bloated after eating too much. The container is stuffed, but not with anything nourishing. How does bloat happen in a real agent? It usually happens gradually, one harmless-looking step at a time. The agent runs a tool, and the tool returns a large block of text. The harness faithfully appends the whole block to the conversation. The agent runs another tool, and again the whole result is appended. After twenty such steps, the history may be dominated by long outputs that were useful for one moment and are useless now. Here are the most common sources of bloat. First, verbose tool output, such as a full log file or a full web page, when only a single line or paragraph mattered. Second, repeated content, such as the same file being read three times because the agent forgot it had already seen it. Third, long pleasantries and restated plans, where the model keeps writing paragraphs of explanation that no later step needs. Fourth, stale items, meaning information that was true earlier but has since been replaced, such as the old version of a file the agent has already edited. Fifth, oversized instructions, meaning a system prompt that has grown to thousands of tokens because every past problem was patched by adding another rule. Bloat has real costs. It raises the price of every call, it slows every call, and as we will see next, it can reduce accuracy. The cure for bloat is discipline in what gets put into the window to begin with, and a regular practice of trimming and summarizing, which you will study in Chapters 20 and 21.