SCORING METHODS

HOW IT WORKS A separate model, the judge, is given the question, the golden answer if one exists, the agent's answer, and clear instructions called a rubric. A rubric is a written scoring guide that describes what each score level means, like the grading guide a teacher uses for essays. The judge then produces a score, for example from 1 to 5, together with a short written justification. The justification matters because it lets a human check whether the judge's reasoning was sound. WHY IT IS POPULAR The judge can read free-text answers, understand that two differently worded answers mean the same thing, and score thousands of outputs in minutes, which would take people days. A GOOD RUBRIC A good rubric is specific. Instead of "score quality from 1 to 5," it says that a score of 5 means the answer is correct, complete, cites the policy, and contains no invented facts, while a score of 3 means the answer is mostly correct but misses one required detail, and a score of 1 means the answer is wrong or unsafe. It is also best to score one dimension at a time, such as correctness, completeness, and policy compliance, rather than asking for one vague overall impression.