REPLAY AND DEBUGGING

REPLAY IN THE PRACTICE LAB The lab's log keeps the messages exchanged with the stand-in model and the tools, which is enough for a simple replay. In Chapter 69 you will use the command that re-runs a single resume, and you will compare the new report with the old one. When the two differ, ask whether the difference comes from the harness, from the data, or from the model. Answering that question calmly, with the records in front of you, is the heart of debugging an agent. REASONING PROBLEMS AND HARNESS PROBLEMS It helps to sort every failure into one of two bins. A harness problem is caused by your software, such as a missing permission, a bad tool description, or a loop with no limit. A reasoning problem is caused by the model's judgment, such as a wrong plan or a misread document. Harness problems are fixed by changing code and are usually repeatable. Reasoning problems are improved by changing the prompt, the context, the examples, or the model, and they may only be reduced and not eliminated. Sorting correctly saves time, because the tools you use for each bin are different.