APPENDIX H: RESUME PORTFOLIO BUILDING CAPSTONE PROJECTS

PROJECT OVERVIEW When a student takes a test, the teacher does not just say "that seemed fine." The teacher uses an answer key, marks each question, adds up a score, and writes comments on the report card. A harness deserves the same treatment, and building the report card is a skill that many employers value, because anyone can build a demo that works once, but few people can prove that a system keeps working. Project 4 builds the teacher. You will create a small target harness, which is the thing being graded, and four versions of a mock model that behave differently: one careful, one lazy, one sneaky, and one that never finishes. You will build a tracer that records spans, where a span is one timed step of a run such as a model call or a tool call, and a trace is the full set of spans for one run. You will build a dataset of ten test cases, a scorer that checks five things about each run, a runner that grades a whole model and writes the results, a quality gate that fails when too few cases pass, and a dashboard page that compares the versions. The project relates to Chapters 27 through 35 and to Chapter 68, and to the fourth stage of the Practice Lab, which built tracing, a fifteen-item evaluation set, and a dashboard. It is different from the lab because it grades several model versions against one dataset and compares them, and because the target harness runs entirely in memory so that the whole evaluation takes a moment. It builds on the ideas of Projects 1 through 3: the target harness is a small cousin of Project 1, with the same think, act, and observe loop and the same folder rules. Use only the invented dataset this project generates, and never real personal data.