BUDGETS AND RESOURCE LIMITS

LEARNING GOALS By the end of this chapter you will be able to describe token, time, and money budgets, explain per-agent and per-workspace budget tracking, describe what should happen when a budget runs out, and prevent runaway loops and infinite retries. WHY AN AGENT NEEDS A BUDGET Imagine giving a teenager a credit card with no limit and no instructions about when to stop shopping. Most teenagers would be sensible, but the risk of a very large bill is real. An agent is similar. An agent works in a loop, and a loop that never ends can spend money, time, and computing power without anyone noticing. A budget is a limit on how much of a resource a run is allowed to use. The harness, not the model, enforces the budget, because a model that is confused cannot be trusted to stop itself. THREE KINDS OF BUDGET TOKEN BUDGETS Recall from Chapter 2 that a token is the small chunk of text a language model reads or writes. You met tokens in Chapter 2. A token budget is a cap on the total number of tokens a run may consume. Think of it as the fuel tank of a car. At every step the harness subtracts the tokens used from the tank and knows exactly how much fuel remains. TIME BUDGETS A time budget is a cap on how long a run may take, measured by the clock on the wall. Some tasks are useless if they arrive late, such as answering a customer who is waiting in a chat window. A time budget protects both the user and the system.