LOGGING, METRICS, AND DASHBOARDS

PRACTICE LAB CONNECTION Build a simple dashboard from the Resume Review Agent lab run logs in Chapter 68. Test the prompts below against it. ARTIFICIAL INTELLIGENCE ASSISTANT PROMPTS [START OF PROMPT] Design a structured logging JSON schema for our harness (fields: timestamp, agent identifier, event type, tool name, duration, cost, status) and validate it against this sample of current log output: [PASTE LAB OR PRODUCTION LOG SAMPLE HERE]. Flag any field in the current logs that is inconsistent, missing, or not machine-parseable. Output format: the schema definition plus a validation findings list. [END OF PROMPT] [START OF PROMPT] Here is six months of a key metric for our agent: [PASTE METRIC TIME SERIES / DASHBOARD DATA HERE]. Determine whether this shows concerning drift versus normal variation, and if drift is present, propose a specific alerting threshold and window that would have caught it early without triggering excessive false alarms on day-to-day noise. Output format: drift assessment, then an alert-rule specification (threshold, window, condition). [END OF PROMPT] CHAPTER SUMMARY Structured logs use consistent labeled fields that programs can read. Key metrics are success rate, latency, cost, token usage, and error rate. Dashboards show metrics live, and well-tuned alerts notify people of problems. Drift monitoring catches slow changes before they become failures.