LATENCY OPTIMIZATION

LEARNING GOALS By the end of this chapter you will be able to explain why speed matters to the people who use an agent. You will be able to measure response time and name ways to reduce it, such as running steps in parallel and streaming results. You will understand why caching plans and using firm timeouts keep an agent responsive. WHY SPEED MATTERS Latency, as you learned, is the delay between asking and receiving. Users judge an agent by how fast it feels. An answer that arrives in two seconds feels like a conversation. An answer that takes a minute feels like a broken page. Speed also affects cost indirectly, because long-running work occupies expensive resources longer. This chapter teaches four skills: measuring and reducing response time, using parallelism and streaming, caching plans and intermediate results, and failing fast with aggressive timeouts.