COST OPTIMIZATION TECHNIQUES

PROMPT CACHING Prompt caching means reusing repeated context to save cost and time. Here is the idea in plain terms. Many calls in an agent session begin with the exact same text: the system prompt and the tool definitions, which are the descriptions of the tools the agent may use. Without caching, the provider processes this same text from scratch on every call and bills you the full input price each time. With caching, the provider remembers the processed form of that unchanging beginning after the first call. Later calls that start with the identical text are charged a much lower rate for that portion and often start faster. It is like a chef who prepares the soup base once in the morning and reuses it all day, rather than chopping onions for every bowl. Several practical rules apply. The cached portion must be an exact match, usually at the start of the prompt, so put stable content first and changing content last. Anything that changes, such as the current date written at the top or a session identifier, breaks the match for everything after it, so keep volatile details near the end. Caches typically expire after a short time if unused, so they help most in busy sessions or repeated calls. Providers differ in how they price cache writes and cache reads, in how long caches last, and in whether you must mark the cacheable parts or it happens automatically. Always read your provider's current documentation for the exact rules, since these details change. The benefit grows with the size of the repeated block and the number of calls that share it.