PREPARING FOR CERTIFICATION

HOW TO TELL REASONING, TOOL, AND CONTEXT PROBLEMS APART This diagnosis skill is worth practicing slowly. Consider three small trace excerpts. In the first, the model asks to read a file that does not exist, and the tool returns a clear error, but the model's next step repeats the same request. The tool behaved correctly, the context contains the error, and the model ignored it, so the root cause is reasoning, and the harness fix is a repeated-call guard. In the second, the model asks for a correct file, but the tool returns an empty string because of a bug in the tool, and the model then reasons sensibly from nothing. The model did well with what it saw, so the root cause is the tool. In the third, the correct result appears early in the trace, but by step thirty the model asks for the same file again as if it had never seen it, and the context at that step no longer contains the earlier result. The model and tool both worked, but the context builder dropped the fact, so the root cause is context management. The questions to ask are always the same. What did the model see at the moment of the bad decision? What did the tool actually return? Given those two things, was the model's choice reasonable?