LATENCY OPTIMIZATION

PRACTICE LAB CONNECTION In the Resume Review Agent lab, the batch-processing step reviews many resumes in one go. In Chapter 69 you will add parallel execution so several resumes are processed at once, with a cap on simultaneous work, and add timeouts to each external call. Record the total time before and after, and use the prompts below against the traces. ARTIFICIAL INTELLIGENCE ASSISTANT PROMPTS Prompt 1: Latency breakdown. Use this when a run feels too slow and you need to find the real cause. [START OF PROMPT] Break this slow run's total latency down into model inference, tool-call, network, and queueing time, and identify the single highest-leverage fix: [PASTE LAB OR PRODUCTION TRACE WITH TIMING DATA HERE]. Output format: a latency breakdown table (Component, Time, % of Total) plus a one-paragraph highest-leverage recommendation. [END OF PROMPT] Prompt 2: Streaming strategy. Use this when designing what the user sees while the agent works. [START OF PROMPT] Design a streaming strategy for this multi-step agent interaction, specifying exactly which parts should stream live to the user versus be held back until fully ready: interaction = [PASTE LAB OR PRODUCTION INTERACTION DESCRIPTION, e.g. "batch resume processing with a final report"]. Output format: a step-by-step streaming plan (Step, Stream Live? Y/N, Justification). [END OF PROMPT]