THE EVALUATION HARNESS

HOW THE TWO WORK TOGETHER The two styles are partners, not rivals. Offline evaluation is the gate you pass through before release, and online evaluation is the security camera that watches after release. A well-run team also feeds the two into each other. When online monitoring finds a new kind of failure, the team turns that failure into a new offline test case, so the dress rehearsal gets harder and smarter over time. A good release plan also defines a rollback rule in advance. A rollback means returning to the previous working version. For example, the plan might say that if the online success score drops more than five points below the offline baseline for a full day, the team automatically switches back to the old version. Deciding that number ahead of time prevents panicked arguments in the middle of an incident.