REPLAY AND DEBUGGING

LEARNING GOALS By the end of this chapter you will be able to define replay, explain how to reproduce failures reliably, perform root-cause analysis, and tell network and tool problems from reasoning problems. WHAT REPLAY IS Replay means re-running a recorded session to study it. Sports teams watch recorded games to see exactly where a play went wrong. An engineer replays a recorded agent run for the same reason. Replay is done offline, in a safe copy, so no real emails are sent and no real data is changed. REPRODUCING FAILURES RELIABLY A failure you cannot reproduce is very hard to fix. Agents make this harder, because a live re-run may meet a changed database, a new web page, or a different random choice by the model. So the harness records everything needed for a faithful replay. That means the user's message, every tool response the agent saw, the timestamps, the model settings such as temperature, and a random seed when the model supports one. Temperature is a setting that controls how random a model's word choices are, and a seed is a starting number that makes random choices repeatable. During replay, the recorded tool responses are played back instead of calling the real tools, so the run unfolds the same way. Even then, models can vary slightly, so replay shows what the agent saw, which is usually enough to understand the failure.