LOGGING, METRICS, AND DASHBOARDS

LEARNING GOALS By the end of this chapter you will be able to define structured logging, name key agent metrics, describe dashboards and alerts, and explain drift monitoring. STRUCTURED LOGGING A log is a written record of events. A free-text log line such as "the agent searched the web and it took a while" is readable by a person but hard for a program to use. Structured logging means writing logs in a consistent, machine-readable format, with the same named fields every time. Think of the difference between a handwritten note and a filled-in form. A form has labeled boxes, so a program can sort, count, and search it. A good log entry has a timestamp, an agent identifier, an event type, a tool name, a duration, a cost, and a status. Many teams use a format called JSON, a text format of labeled fields, for this purpose. KEY METRICS A metric is a number measured over time. The most useful agent metrics are: Success rate, the share of runs that meet their success criteria. Latency, the time a run takes from request to answer. Cost, the money spent per run. Token usage, the number of tokens consumed per run. Error rate, the share of tool calls that fail. Watching them together tells a story. If success falls while latency and cost rise, something upstream, such as a model or a tool, has probably changed.