COST OPTIMIZATION TECHNIQUES

CHECK YOUR UNDERSTANDING • Why is a long system prompt especially costly? ◦ Answer: It is sent and billed on every call, so each extra word is paid for repeatedly. • What is model routing, and what safety net should accompany it? ◦ Answer: Sending easy tasks to cheaper models and hard tasks to stronger ones, with escalation to a stronger model if the cheap one fails a check. • Why should changing details such as the date go at the end of a prompt? ◦ Answer: Caching needs an exact match of the beginning, and a changing detail near the start would break the match. • Name two sources of wasted tokens and a fix for each. ◦ Answer: Redundant file reads, fixed by reading once and reusing; blind retries, fixed by limited retries with waiting. • Name two techniques for lowering the cost of an agent without changing what it does. ◦ Answer: Any two of prompt compression, model routing, prompt caching, and cutting wasted tokens from loops and retries.