RUNTIME EXECUTION ENVIRONMENTS

BATCH AND PARALLEL EXECUTION So far we have pictured one action at a time. But many tasks contain pieces that do not depend on each other. Batch execution means grouping many similar items and processing them as one job, such as reviewing 200 resumes in one run. Parallel execution means running several tasks at the same moment rather than one after another. Imagine five cashiers serving five customers at once, instead of one cashier serving all five in a row. For an agent, parallelism usually means issuing several independent tool calls at the same time. Many model providers allow a model to request multiple tool calls in a single reply, and the harness then runs them together and returns all results. The benefit is wall-clock time. Wall-clock time means the real time that passes on a clock on the wall, as opposed to the amount of computer work done. If five searches each take 10 seconds, doing them in order takes about 50 seconds, but doing them together takes roughly 10. There are cautions. Only run steps in parallel when they are truly independent, meaning one does not need the output of another. Parallel work can hit rate limits sooner, because the outside service sees many requests at once. Parallel tools that change the same file or record can collide, so the harness may need locks, which are rules that let only one worker change a thing at a time. Parallel results arrive in unpredictable order, so the harness should label each result with the request it belongs to. Finally, parallel steps can multiply costs quickly, so they still fall under the step and budget limits you met in Chapter 9 and will study more in Part Six.