BENCHMARKS AND LEADERBOARDS

THE LIMITS OF BENCHMARKS Benchmarks are tools, not truth. Several limitations deserve your attention. Limit one: mismatch with your real work A public benchmark measures what its designers chose to measure. Your resume-review agent or support agent may face very different inputs, policies, and tools. A top score on a coding benchmark does not guarantee success on your internal documents. Limit two: contamination Contamination happens when benchmark questions or answers leaked into the data used to train a model. If a student saw the exam in advance, a high score proves little. Because public benchmarks live on the internet, contamination is a real concern. Limit three: saturation A benchmark is saturated when top systems score so close to the maximum that the test can no longer tell them apart. Limit four: narrow metrics A single accuracy figure may hide cost, speed, and safety. Limit five: gaming and teaching to the test Teaching to the test means tuning your system specifically to score well on a known benchmark, rather than to be good in general. A student who memorizes the practice exam answers can score perfectly and still not understand the subject. There is a famous idea related to this, often summarized as: when a measure becomes a target, it stops being a good measure. That idea is known as Goodhart's Law. In harness engineering, the risk looks like this. A team repeatedly tweaks its harness against the same public benchmark questions. Scores rise, but real-world user satisfaction stays flat or falls, because the benchmark no longer reflects general performance.