SCORING METHODS

PRACTICE LAB CONNECTION In the Resume Review Agent lab (Chapter 68), you will build an LLM-as-judge scoring pipeline. You will write a rubric for the lab agent's match assessments, run a judge over the sample outputs, and compare the judge's scores to your own scores. Test the prompts below against that rubric and those sample outputs. ARTIFICIAL INTELLIGENCE ASSISTANT PROMPTS [START OF PROMPT] Act as the LLM judge and score these agent outputs against the golden reference, using a 1-to-5 scale with written justification, and flag any case where you suspect your own judgment might be biased (position bias, verbosity bias, shared blind spot with the agent model): question = [TEST QUESTION], golden answer = [GOLDEN ANSWER], candidate answers = [PASTE LAB OR PRODUCTION CANDIDATE ANSWERS HERE]. Output format: a score table (Candidate, Score, Justification, Bias Flag if any). [END OF PROMPT] [START OF PROMPT] Design an LLM-as-judge scoring rubric for this agent's outputs on these dimensions: [DIMENSIONS, e.g. correctness, completeness, policy compliance], with a 1-to-5 scale and a one-sentence description per level per dimension. Then apply the rubric to these two sample outputs for the same input and show the scores: [PASTE TWO SAMPLE OUTPUTS HERE]. Output format: the rubric table, then the applied scores with justification. [END OF PROMPT]