PROJECT FOUR - EVALUATION AND OBSERVABILITY

CHAPTER SUMMARY Chapter 68 gave the harness instruments and a test. Spans record each timed piece of work, a trace gathers the spans of one task, and a dashboard summarizes the health of many runs. A runaway loop shows up in traces as one identical span repeated many times, and a guard that stops the third identical call in a row repairs it. An evaluation set of fifteen golden-answer cases, run after every change, turns quality from a feeling into a pass rate. In our test run the guard cut the empty-file case from ten steps to six and lifted the pass rate from fourteen of fifteen to fifteen of fifteen. KEY TERMS REVIEW Observability: the ability to understand a system from the records it emits. Span: the record of one timed piece of work. Trace: all the spans belonging to one task. Dashboard: a one-view summary of system health. Runaway loop: a loop that repeats a useless action until stopped from outside. Guard: a harness rule that stops or redirects bad behavior. Evaluation set: a fixed collection of test tasks with known good answers. Golden answer: the correct expected result for an eval item. Normal, edge, and adversarial cases: the three kinds of eval item. Pass rate: the share of eval items that passed.