WHY HARNESS ENGINEERING EMERGED AS ITS OWN FIELD

EXAMPLES Example 1: From single answers to multi-hour tasks. In 2022 and 2023, a typical request to a chatbot was, "Explain how photosynthesis works." The system replied in a paragraph and the interaction was over. By 2025 and 2026, a typical agent request might be, "Research the top ten competitors in the electric bicycle market, read their websites and recent news, and produce a report with sources." To complete that task, the agent might make dozens of searches, read dozens of pages, take notes, discard irrelevant material, and assemble a draft. Walk through what can go wrong: the agent might read the same page twice, forget what it had already concluded, cite a source it never opened, or spend far more money than expected. Each of those problems requires a harness feature, such as a list of visited pages, a notes file, a citation check, and a spending cap. The jump in task length is what created the need. Example 2: Failures traced to context and tools. A company runs a coding agent and reviews one hundred failed runs. They sort the failures by cause. In many of them the agent did not have the right file in view. In others a tool, such as the one that searches the codebase, returned too many results and flooded the context. In others the tool returned an error written in a way the model misread as success. Relatively few failures were due to the model being unable to understand the problem. The team concludes that improving how context is selected and how tools report results will help more than waiting for a better model. This kind of finding is a typical source of the claim that launched harness engineering as a distinct discipline. Treat the numbers in any specific company's report as that company's experience, not as a universal law. Example 3: Agents writing much of the code. Some teams report that a large share of their production code now comes from agents, while human engineers set direction, write specifications, review changes, and handle the hardest design decisions. Consider how such a team might work in practice. A human writes a short description of a feature. An agent proposes a plan, the human approves it, and the agent writes the code and the tests. Automated checks run, and a human reviews the final change before it is merged. Notice that the team's success depends less on the agent being brilliant than on the system that checks the agent's work. Such reports are encouraging, but they should be read with care since they are rarely independently verified. Example 4: An automated experiment on context compaction. Compaction means shrinking a long history by summarizing older parts, which you will study in Part Five. An internal evaluation pipeline is built to compare two compaction strategies. Strategy A summarizes older messages once the history passes forty thousand tokens. Strategy B summarizes once it passes eighty thousand tokens. The pipeline runs the same fifty tasks under each strategy, three times each, and records success rate, average cost, and average time. Suppose Strategy A succeeds on forty-one tasks with lower cost, and Strategy B succeeds on forty-three tasks at higher cost. The pipeline applies a rule written in advance: promote the cheaper strategy unless the other is better by at least five percentage points in success. Since the difference is only four points, Strategy A is promoted. The underlying model was never touched. Only the harness setting changed, and the decision was made by a fixed rule rather than by a hunch.