COST OPTIMIZATION TECHNIQUES

PROMPT COMPRESSION A prompt is the text you send to the model, including instructions, examples, and background information. The system prompt is the standing set of instructions that is sent at the start of every call, telling the model its role and rules. Because the system prompt is sent on every single call, every word in it is paid for again and again. Prompt compression means rewriting prompts to use fewer tokens while keeping the same meaning and behavior. Imagine you are leaving a note for a house sitter. A long rambling note with repeated reminders and polite filler takes the sitter longer to read and does not make the sitter more reliable. A short note that says "Feed the cat at 8 and 6, water the plants on Sunday, call Maria for emergencies" is faster to read and just as clear. Ways to compress a prompt include the following. Remove repetition: if a rule is stated three times in different words, state it once clearly. Remove filler: phrases like "I would really like you to please try to" add tokens but no information. Use structure: short numbered rules are often clearer than long paragraphs. Trim examples: keep the one or two best examples rather than ten similar ones. Shorten tool descriptions: describe each tool in one or two precise sentences. Trim context sent along: do not include whole documents when a relevant excerpt will do, and summarize old conversation history instead of resending every word. Compression has a risk. If you remove a rule that mattered, behavior changes. So never compress without a test. Run your evaluation suite, which is a collection of test cases with known good answers, before and after the change. If scores hold steady and cost drops, keep the change. If scores drop, restore what you cut.