← Table of Contents · Page 47 of 868

THE THREE DISCIPLINES AROUND THE MODEL

Part 6 of 11

WHY THE HARNESS OFTEN MATTERS MORE THAN THE MODEL You may hear the claim that in real systems the harness often matters more than the model. It is worth understanding both what it means and what it does not mean. It does not mean that the model is unimportant. A weak model will fail at tasks that need strong reasoning, whatever surrounds it. It means that, above a certain level of model capability, the differences between how well two real systems perform often come from the surrounding engineering rather than from small differences between models. Several reasons support this. First, many real-world failures are not failures of reasoning at all. They are failures of context, where the model was not shown the needed fact, or of tooling, where a tool returned a confusing result, or of control, where the loop ran too long or stopped too soon. Second, models change quickly. A new model arrives every few months, but a well-designed harness lets you swap it in and keep working. The harness is the lasting investment, while the model is a replaceable part. Third, reliability is a property of the whole system. Even a very capable model that is right ninety-five percent of the time per step will fail often over a long chain of steps unless errors are caught. A harness that verifies and corrects after each step turns an unreliable chain into a dependable one. Fourth, cost and safety are controlled by the harness. The model cannot enforce its own spending limit or its own permissions. Reported experience from teams running coding agents in production points in this direction: they often found that most failures traced to context handling and tool design rather than to the model's raw reasoning. Treat such reports as indications, not as laws of nature, but they match the logic above. Keep the claim modest and precise. The sound statement is that a good harness lifts the performance of any capable model, and that investing in the harness is often the most reliable way to improve a real system.
Advertisement
Read Aloud:
0:00 / 0:00