← Table of Contents · Page 365 of 868
TESTING THE HARNESS ITSELF
Part 2 of 12
WHAT TO UNIT TEST IN A HARNESS
Tools: every tool should have tests for normal inputs, bad inputs, and edge cases.
Parsers: the code that reads model output, such as pulling a tool name and arguments out of text, should be tested with well-formed and malformed examples.
Permission resolver: the code that decides whether an action is allowed should be tested with allowed, denied, and tricky requests.
Context builder: the code that assembles what the model sees should be tested to confirm it includes the right items, respects length limits, and drops the right items when space runs out.
Retry and error logic: tests should confirm that failures trigger the intended recovery.
USING FAKES AND MOCKS
A key technique is to replace risky or slow outside things with fakes.
A mock, or fake, is a stand-in object that behaves like the real thing in a controlled way.
For instance, a test for an email tool should never send a real email.
Instead, the test uses a fake mail service that simply records what it was asked to send, so the test can check the request without any real-world effect.
Fakes also make tests deterministic, meaning they give the same result every time, which matters because real services and models can be slow, costly, or random.
WHAT MAKES A GOOD UNIT TEST
A good unit test checks one behavior, has a clear name that describes the behavior, is independent of other tests, and checks both the happy path (everything goes well) and unhappy paths (bad input, failures).
Advertisement
Read Aloud:
0:00 / 0:00