COST OPTIMIZATION TECHNIQUES

CUTTING WASTED TOKENS FROM LOOPS, RETRIES, AND REDUNDANT CALLS The cheapest token is the one you never send. Waste usually hides in places like these. Redundant reads: the agent fetches the same file or page again on each step. Fix: read once, keep the result in working memory or a cache, and refer to it. Runaway loops: the agent repeats the same failing action over and over. Fix: loop detection and step limits, which you will study in Chapter 56. Blind retries: a failed call is retried immediately and endlessly, each retry paying for the full prompt again. Fix: limited retries with sensible waiting. Bloated tool results: a tool returns a ten-thousand-word web page when the agent needed one sentence. Fix: have the tool trim, filter, or summarize its output before it enters the conversation. Unneeded history: old, irrelevant steps stay in the conversation forever. Fix: summarize or drop what is no longer needed. Over-calling: the agent asks the model to do things that ordinary code could do, such as adding numbers or sorting a list. Fix: let the harness do deterministic work in regular code, which is free and exact. A habit worth building is to look at a trace of a typical run and ask of every model call whether it was truly necessary.