REPLAY AND DEBUGGING

ROOT-CAUSE ANALYSIS Root-cause analysis means asking why repeatedly until you reach the underlying reason, not just the visible symptom. A wrong answer may turn out to be correct given the data the agent was given. If a tool returned outdated numbers, the true root cause is the stale data source, not the model. A good analysis ends with a fix and with a new regression test, a test that makes sure the same bug does not come back, as discussed in Chapter 35. NETWORK AND TOOL PROBLEMS VERSUS REASONING PROBLEMS The first question in debugging is where the fault lies. A tool problem shows in the trace as an error, such as a HTTP 503 status, which means the service was unavailable. A reasoning problem shows as a tool that worked perfectly while the model still drew a wrong conclusion from correct data. The fixes differ. Tool problems call for retries, fallbacks, or repairs to the service. Reasoning problems call for better instructions, clearer context, or added checks. Guessing wrong wastes time, which is why the trace evidence matters.