LATENCY OPTIMIZATION

CACHING PLANS AND INTERMEDIATE RESULTS In Chapter 54 you met prompt caching, a provider feature. Here we talk about caching at the harness level. A cache is a place that stores the result of earlier work so it can be reused instead of recomputed. Plans can be cached. If your agent frequently receives similar requests, such as "summarize this week's sales," it might save the plan it produced last time and reuse it, skipping a model call for planning. Intermediate results can be cached too. If a tool call such as "get the exchange rate" returns the same answer for the next hour, store it for an hour. If a large document was already parsed, keep the parsed version. Caching raises a classic risk: stale data, meaning information that was true when saved but is no longer. Every cache needs a rule for when its contents expire, often called a time to live, and a way to clear it when the underlying data changes. A cache for stock prices should expire in seconds, while a cache for a company's founding year might never need to expire.