BENCHMARKS AND LEADERBOARDS

HOLISTIC LEADERBOARDS A leaderboard is a public ranking table that lists systems by their scores, like a sports league table. Some leaderboards rank by a single benchmark. A holistic leaderboard instead tries to evaluate systems across many scenarios and many measures at once, such as accuracy, fairness, robustness, efficiency, and safety, so that no single number dominates. The word holistic means looking at the whole rather than a single part. The motivation is simple. A system that scores highest on one narrow test may be slow, expensive, or unsafe, and a single number hides those tradeoffs. Examples of such broader efforts include academic projects that evaluate language models across many scenarios and metrics, and community platforms where people vote on which of two anonymous model answers they prefer. Leaderboards are useful for getting a general sense of the field, but they are not a substitute for testing on your own tasks.