← Table of Contents · Page 303 of 868
LOGGING, METRICS, AND DASHBOARDS
Part 7 of 9
SERVICE LEVELS IN PLAIN WORDS
Teams often set a target for how well a system should perform, such as "ninety-five percent of runs succeed and ninety-five percent finish within thirty seconds."
Such a target is called a service level objective.
The metric that measures it is a service level indicator.
Having a clear target turns arguments about whether the agent is good enough into simple comparisons of a number against a goal.
It also tells you when to stop adding features and spend time on reliability, namely when the numbers fall below the target.
EXAMPLES
Example 1: Structured versus free text
A structured entry records the timestamp, the agent identifier, the event "tool_call", the tool "search_web", and a duration of 420 milliseconds.
A program can count all slow search calls instantly, which it cannot do with a free-text sentence.
Example 2: Three metrics together
In one week success rate fell from ninety-four to eighty-one percent, latency rose from three to eleven seconds, and cost per run rose from two cents to nine cents.
Together they suggest an upstream model or tool change.
Example 3: An alert
An alert fires when the success rate stays below a set limit for fifteen minutes.
An on-call engineer is paged before most users notice.
Example 4: Drift
Average response length has grown by forty percent in three months with no code change.
Drift monitoring flags it for investigation even though nothing is technically broken.
Advertisement
Read Aloud:
0:00 / 0:00