LATENCY OPTIMIZATION

CHECK YOUR UNDERSTANDING • Why measure before optimizing? ◦ Answer: Because the slowest component is often not the one you expect, and effort spent on a small slice brings little benefit. • When can tasks safely run in parallel? ◦ Answer: When they do not depend on each other's results, and when running them together will not overload a service or cause conflicting changes. • Does streaming make the model finish faster? ◦ Answer: No, it only makes the wait feel shorter by showing output as it arrives. • What danger comes with caching? ◦ Answer: Stale data, which is why caches need expiry rules. • What should happen after a timeout? ◦ Answer: A retry, fallback, or clear message to the user, not just silence.