TESTING THE HARNESS ITSELF

MAKING INTEGRATION TESTS RELIABLE Because real models vary, teams often script the model's side too. They create a scripted fake model that returns a prepared sequence of responses, plus mocked tool responses, and then verify that the harness handled the whole sequence correctly: called the right tools in the right order, passed results back, stopped at the right time, and produced the right final state. This approach tests the harness code itself, separate from the model's intelligence. Some teams also run a smaller number of integration tests against a real model, accepting some variation, to catch problems that only appear with real model behavior. Integration tests are slower and a failure points to a larger area of code, so you write fewer of them than unit tests. A common picture is a testing pyramid: many fast unit tests at the bottom, fewer integration tests in the middle, and a small number of broad end-to-end and evaluation runs at the top.