PROJECT FIVE - MULTI-AGENT AND PRODUCTION HARDENING

THE PRODUCTION-READINESS CHECKLIST A production-readiness checklist asks, item by item, whether a system is fit for real use. The value of the list lies in honesty. A gap that is written down is a task, while a gap that is hidden is a future incident. Here is a completed checklist for the finished lab, written the way we want you to write yours. Each line gives the item, its status, the evidence, and the remaining work. Evaluation suite: Partial. Evidence: fifteen items with a documented pass rate. Remaining: more cases and testing against a real model. Budgets and limits: Done. Evidence: step limits per subagent and a batch budget in routing.py. Remaining: none for the lab. Logging: Done. Evidence: run_log.jsonl, security_log.jsonl, and traces.jsonl. Tracing and dashboard: Done. Evidence: spans on every call and a dashboard of the last twenty-five runs. Alerting: Missing. Evidence: the dashboard must be opened by a person. Remaining: automatic warnings on repeated spans or failures. Least-privilege permissions: Done. Evidence: the permission gate and the tool allow-list. Human approval for risky actions: Done. Evidence: the approval hook with a logged decision. Prompt injection defense: Partial. Evidence: untrusted labeling, line removal, and a passing eval item. Remaining: broader attack tests and a formal review. Tool supply-chain checks: Partial. Evidence: description validation on the MCP tool. Remaining: origin checks and scanning for real servers. Retries and loop protection: Done. Evidence: backoff, a repeated-call guard, and step limits tested by deliberate failure. Secrets handling: Partial. Evidence: keys are meant to come from environment variables. Remaining: a scan that proves no key is ever saved. Rollback plan: Partial. Evidence: version-control checkpoints per stage. Remaining: a rehearsed procedure for a deployed system. Human escalation path: Partial. Evidence: the supervisor stops and names the failure. Remaining: a real person or channel who receives it. Testing with a real model: Missing. Evidence: only the mock model was exercised. Remaining: run the whole suite against a real model and repeat several times. Documentation and runbook: Partial. Evidence: this chapter and your notes. Remaining: a short guide for what to do when it breaks. Final statement. For learning and for the sandbox the lab is a go. For a real deployment handling real people's resumes it is a no-go until alerting, real-model testing, escalation, and a security review are done, and that is a perfectly respectable result for a capstone project.