CONTEXT AS THE SCARCE RESOURCE

CHAPTER SUMMARY The context window is the agent's working memory, comparable to the RAM in a computer. It holds system instructions, tool descriptions, history, tool results, retrieved documents, and the room the model needs to answer. In the operating-system analogy, the model is the processor, the window is the memory, storage is the disk, and the harness context builder is the memory manager. Context is finite because of model limits, cost, speed, and quality. A good harness budgets context in advance, reserves room for the answer, and decides for every item whether to include it, summarize it, or exclude it. KEY TERMS REVIEW Context window: the maximum amount of text, measured in tokens, that a model can consider in one call. Token: a small chunk of text, roughly three quarters of an English word on average. Context builder: the harness component that assembles the text the model sees on each call. Budget: a planned allowance of a limited resource for each purpose. Retrieval: searching stored documents and pulling out only the relevant pieces. Eviction: removing items from the window to make space.