APPENDIX H: RESUME PORTFOLIO BUILDING CAPSTONE PROJECTS

WHAT YOU WILL BE ABLE TO CLAIM AFTERWARDS You built an evaluation suite for an agent harness with a dataset, an automated scorer, and a regression gate that fails with a nonzero exit code. You instrumented a harness with traces that record the duration and token estimate of every model and tool call. You compared four model versions on the same ten cases and produced a dashboard that shows pass rates and the reasons for failures. You can explain the difference between a metric, a trace, and a log. Do not claim that the scores measure real model quality, because the models are scripted and the cases are tiny and invented. SKILLS PRACTICED Why observability matters, covered in Chapter 27. Tracing and spans, covered in Chapter 28. Logging, metrics, and dashboards, covered in Chapter 29. Replay and debugging, covered in Chapter 30. The evaluation harness, covered in Chapter 31. Designing evaluations, covered in Chapter 32. Scoring methods, covered in Chapter 33. Testing the harness itself, covered in Chapter 35. Cost thinking, covered in Chapter 53.