← Table of Contents · Page 697 of 868

PROJECT FOUR - EVALUATION AND OBSERVABILITY

Part 6 of 9

PRACTICE LAB CONNECTION Stage 4 deliverables. Tracing on every model and tool call, a dashboard summarizing the last twenty-five runs, and a fifteen-item eval_set.json with a scoring pipeline. Intentional bug to find. The runaway loop triggered by resume_empty.txt, found through the repeated-span pattern and fixed with the repeated-identical-call guard. Acceptance criteria for Stage 4. First, all fifteen items run through the scoring pipeline with a documented pass rate. Second, the runaway loop is caught by tracing, visible as a repeated-span pattern, and fixed, with a before and after trace showing the fix working. Third, the dashboard correctly reflects the change in cost and step count between the before run and the after run. Make a version-control save labeled stage4.
Advertisement
Read Aloud:
0:00 / 0:00