← Table of Contents · Page 697 of 868
PROJECT FOUR - EVALUATION AND OBSERVABILITY
Part 6 of 9
PRACTICE LAB CONNECTION
Stage 4 deliverables.
Tracing on every model and tool call, a dashboard summarizing the last twenty-five runs, and a fifteen-item eval_set.json with a scoring pipeline.
Intentional bug to find.
The runaway loop triggered by resume_empty.txt, found through the repeated-span pattern and fixed with the repeated-identical-call guard.
Acceptance criteria for Stage 4.
First, all fifteen items run through the scoring pipeline with a documented pass rate.
Second, the runaway loop is caught by tracing, visible as a repeated-span pattern, and fixed, with a before and after trace showing the fix working.
Third, the dashboard correctly reflects the change in cost and step count between the before run and the after run.
Make a version-control save labeled stage4.
Advertisement
Read Aloud:
0:00 / 0:00