LATENCY OPTIMIZATION

FAILING FAST WITH AGGRESSIVE TIMEOUTS A timeout is a limit on how long you will wait for something before giving up. Without timeouts, one unresponsive service can freeze your entire agent. Failing fast means detecting trouble quickly and moving on, rather than waiting a long time for a result that may never come. If a tool normally answers in 300 milliseconds, waiting 60 seconds for it is pointless. A reasonable timeout might be 3 seconds. After the timeout, the harness can try a retry, switch to a fallback, or tell the user what happened. Set timeouts using real measurements. Look at how long the tool usually takes, perhaps its 99th percentile, and set the timeout a bit above that. Too short, and you give up on calls that would have succeeded. Too long, and you lose the benefit. Remember also to set an overall time limit for the entire run, so that many small waits cannot add up to a very long wait. Failing fast works best with the reliability tools of Chapter 56, such as retries and fallbacks, because a timeout is only the first half of the answer; what happens next matters just as much.