WHY HARNESS ENGINEERING EMERGED AS ITS OWN FIELD

REAL-WORLD EVIDENCE What evidence supports the claim that this discipline matters? Three kinds of evidence are commonly cited, and each deserves a careful look. The first is reports from teams running coding agents in production. Several teams have said that when they investigated why agent runs failed, most of the causes traced back to context management, such as the model not having the needed file or being swamped by old output, and to tool design, such as a tool that returned confusing messages. Fewer failures were attributed to the model being unable to reason about the task. This is a useful signal, because it points engineers toward the harness as the place to invest. The second is reports of substantial software being produced with heavy agent assistance. Some teams have publicly said that large portions of their production code are now written by agents, with human engineers directing the work, reviewing the results, and deciding what to build. Such claims are usually self-reported and difficult to compare, so a careful reader treats them as signs of a trend rather than as measured facts. Still, the pattern in these reports is consistent: the people who succeed describe elaborate surrounding systems of tests, review steps, and permissions, not just a clever model. The third kind of evidence is controlled comparisons. Researchers and companies have run experiments in which the same model is placed in different harnesses and scored on the same tasks, and in many such studies the harness changes the score by a meaningful amount. Public leaderboards for coding agents often list the harness or scaffold alongside the model for this reason. When reading any such comparison, ask three questions. Who ran the test? Was the same set of tasks used? Could the result be partly due to luck, because only a few runs were made? Asking these questions is itself part of harness engineering, because you will constantly need to judge whether a change really helped. An engineer who accepts every claim at face value will be misled, and an engineer who dismisses every claim will miss real progress. The balanced stance is to treat each claim as a hypothesis worth testing in your own system.