COST OPTIMIZATION TECHNIQUES

CHAPTER SUMMARY Optimization means saving money without hurting quality, and every change should be tested. Prompt compression removes repetition and filler from text that is paid for on every call. Model routing matches task difficulty to model size, with escalation as a safety net. Prompt caching lets the unchanging start of a prompt be charged at a lower rate, so put stable content first. Wasted tokens from redundant reads, loops, retries, and bloated tool results are often the easiest savings. KEY TERMS REVIEW System prompt: the standing instructions sent at the start of each call. Tool definitions: the descriptions of the tools the agent can use. Prompt compression: shortening prompts while preserving behavior. Model routing: choosing a model by task difficulty. Escalation: retrying with a stronger model when a cheaper one falls short. Prompt caching: reusing processed repeated context at a reduced rate. Evaluation suite: a set of test cases used to check quality.