← Table of Contents · Page 233 of 868

MEMORY HIERARCHIES FOR CONTEXT

Part 7 of 11

EXAMPLES Example 1: Hot, warm, and cold in an ordinary workday. Imagine a support agent that helps a company's customers over many weeks. Today, a customer is chatting about a delayed order. The current conversation is in the hot tier, the window. Last week's chat with the same customer about an address change was compacted into a three-line summary and stored in a database. That is the warm tier, and one lookup by customer identifier brings it back. The full word-for-word transcript of that earlier chat, including the customer's long explanation, is stored in an archive. That is the cold tier, and it is fetched only if someone explicitly asks for the exact conversation, for instance if there is a dispute. For the delayed order, the warm summary is enough, so the agent never touches the cold tier. The conversation is fast, cheap, and informed. Example 2: Progress notes after compaction. A coding agent builds a new feature over hundreds of steps. It maintains a file named progress.md, which lists completed subtasks, open questions, decisions, and the next action. Midway, the harness compacts the history. Unlike a summary, the file is not lossy, because it has been updated precisely after each step. After compaction, the agent's first action is to re-read progress.md. It sees that the login page and the database setup are finished, that the payment page is in progress, and that an open question remains about which payment provider to use. The agent continues at exactly the right point, with exact status, and without the guessing that a purely summarized history can cause. Example 3: Offloading a huge log. An agent investigating a crash runs a command that prints a 10,000-line log. The harness intercepts the output. It saves the whole log to disk under a file name that includes the run number. It scans the log for lines that look like errors and finds one at line 230. It places into the context only this: "Full log saved to the logs folder as run 42. Key error at line 230: database connection refused. Use the read tool with a line range for more." The agent reads the error, reasons about the cause, and asks for lines 200 through 260 to see what happened just before. Only that small slice enters the window. The total tokens used are a small fraction of what pasting the entire log would have cost. Example 4: Moving between tiers on a follow-up question. A user returns after three days and asks, "Why did we choose the second vendor?" The hot tier is empty because this is a new session. The harness searches the warm tier and finds a summary from three days ago containing the line "Decision: chose Vendor B because Vendor A could not meet the delivery date." That answers the question, so the harness loads only this summary into the window, not the entire transcript. If the user then says, "Show me exactly what Vendor A wrote," the harness goes to the cold tier and retrieves that one message. Each step used only as much storage and effort as the question required.
Advertisement
Read Aloud:
0:00 / 0:00