LOGGING, METRICS, AND DASHBOARDS

LOGGING IN THE PRACTICE LAB The lab writes its events as one line of structured text per event, with the run identifier, the time, the kind of event, and a few details. In Chapter 68 you will also write a small summary script that reads these lines and prints the success rate, the average number of steps, and the average cost. That script is a tiny dashboard in text form. Run it after several experiments and notice how the numbers change when you alter the agent, which is exactly how professional teams judge a change. WATCHING FOR DRIFT Drift means that behavior slowly changes over time without anyone deliberately changing it. A provider may update a model, a tool may start returning slightly different text, or users may start asking different kinds of questions. Because the change is gradual, nobody notices on any single day. To catch drift, compare this week's metrics with a stable baseline and alert when they move beyond an agreed range. Also re-run the evaluation set on a schedule, since a fixed set of tests gives a steady yardstick.