PROJECT TWO - ADDING CONTEXT MANAGEMENT

MEASURING THE EFFECT It is easy to claim that a change helps. It is much harder, and much more valuable, to measure it. A fair measurement changes only one thing, uses the same tasks, and checks both the cost and the quality. Run the comparison command shown below. [START OF BASH CODE] # Run the batch twice, once without and once with the context builder, and compare cost and reports python main_loop.py --compare [END OF BASH CODE] In our own run of this exact code with the mock model, the last lines looked like this, and yours should be very close because the mock behaves the same every time. BEFORE 442351 tokens, AFTER 74472 tokens, saved 83% Reports identical before and after: 25 of 25 Read these numbers carefully and honestly. The first line shows that sending the whole history every time cost far more estimated tokens than sending a tidied context. The second line shows that all twenty-five reports came out the same, so quality did not drop. Because our token count is a rough estimate and our model is a stand-in, the exact percentage is not a promise about real systems. With a real model your savings will differ and the reports may differ a little from run to run, so you would repeat the test several times and look at the average and the spread before believing a result. What matters here is the method: same tasks, one change, cost and quality both checked.