SCORING METHODS

CHAPTER SUMMARY Scoring methods range from rigid to flexible: exact match, rule-based checks, LLM judges, and human review. Exact match and rules are cheap and consistent but brittle or narrow, LLM judges handle open-ended text but have biases and blind spots, and humans are the most trusted but the slowest. Bias is a consistent lean and variance is random wobble; both can be reduced by averaging repeated runs, swapping order, using multiple judges, writing precise rubrics, and calibrating against human scores. Most real evaluation systems layer several methods together. KEY TERMS REVIEW Scoring: deciding how good an answer is. Exact match: checking that output is identical to the expected answer. Normalization: cleaning text into a standard form before comparing. Rule-based scoring: applying explicit coded checks. LLM as a judge: using a separate model to score outputs. Rubric: a written scoring guide describing each score level. Position bias: favoring an answer because of its place in order. Verbosity bias: favoring longer answers. Calibration: comparing a judge against human scores to check trustworthiness. Variance: random wobble in scores.