TESTING THE HARNESS ITSELF

CHAPTER SUMMARY The harness is ordinary software and needs ordinary software testing in addition to agent evaluation. Unit tests check small components in isolation, often using fakes; integration tests check the full loop working together; regression tests check that old behavior still works after changes. Evaluation-driven iteration improves the system by measure, hypothesize, change, and re-measure cycles. Continuous evaluation runs the eval suite automatically in the pipeline and blocks merges that drop quality below an agreed threshold, with a visible override process for justified exceptions. KEY TERMS REVIEW Unit test: an automatic check of one small piece of code. Mock or fake: a controlled stand-in for a real outside component. Integration test: a test that checks components working together. Regression: something that used to work breaking after a change. Regression testing: checking that new changes do not break old behavior. Baseline: the previous recorded score used for comparison. Evaluation-driven iteration: improving a system guided by measured results. Pipeline: an automated assembly line from code change to release. Gate: an automatic checkpoint a change must pass. Pull request: a proposal to merge code changes.