MEMORY HIERARCHIES FOR CONTEXT

WHY A HIERARCHY Look at how your own life handles information. The phone number you are about to dial is held in your head for a few seconds. Your friends' birthdays are written on a calendar on your wall, one glance away. Your tax records from five years ago are in a box in the attic, which takes effort to reach, but they are there if the tax office ever asks. You naturally keep fast, small, important things close, and slow, large, rarely needed things far away. Computers do the same. A computer has tiny, extremely fast storage close to the processor, medium storage in RAM, and huge, slower storage on disks. Engineers call this arrangement a memory hierarchy. A hierarchy is simply a ladder of levels, each larger and slower than the one above. An agent harness should use the same wise structure for context. Instead of treating the window as the only place information can live, we add levels beneath it, so that information leaving the window is not destroyed but stored where it can be found again. THE THREE TIERS: HOT, WARM, AND COLD A common and easy-to-remember design uses three tiers. The hot tier is the active context window. It holds what the model is reading right now. It is the fastest to use, because no lookup is needed, but it is the smallest and most expensive per token. The warm tier holds compressed summaries. These are the compaction summaries from Chapter 21, perhaps along with short notes and key facts, stored in a database or a file. Retrieving from the warm tier takes one lookup, and the content is already small, so bringing it back into the window is cheap. The cold tier holds complete, unabridged transcripts and raw data. This is the full record: every message, every tool result, every file, saved in long-term storage. Retrieving from the cold tier takes more effort and can produce a lot of text, so it happens only on explicit request, when exact detail is truly needed. The names hot, warm, and cold come from the idea of temperature. Hot means in constant use, and cold means resting in the archive. You can compare the three tiers to a restaurant kitchen. The hot tier is the cutting board in front of the chef, holding only the ingredients for the dish being prepared. The warm tier is the small refrigerator under the counter, with prepped ingredients ready within seconds. The cold tier is the big storeroom in the back, with bulk supplies that someone must walk to get. A chef who tried to keep the entire storeroom on the cutting board could not cook at all. A chef who had no storeroom would run out of ingredients. The system works because each level has a role.