DOMAIN-SPECIFIC HARNESSES

EDGE AND ON-DEVICE AGENTS An edge agent, also called an on-device agent, is an agent that runs on local hardware, such as a phone, a laptop, a vehicle, or a machine on a factory floor, instead of in a distant data center. The word edge comes from the idea that the work happens at the edge of the network, close to where the data is produced. Think of the difference between a thermostat that decides by itself in your hallway and one that must phone a faraway office to ask permission each time. Edge agents use small, efficient models that fit the limited memory and power of local devices. The trade-offs are as follows. Speed: no round trip over the network, so decisions can be nearly immediate. Privacy: data can stay on the device and never travel anywhere. Reliability: it keeps working when the connection is poor or absent. Cost: no per-request fee to a hosted model, though the device itself costs money and energy. Capability: small models are generally less capable than the largest hosted ones, so they handle narrower tasks. Updates: improving or fixing software spread across thousands of devices is harder than updating one server, which links to the deployment ideas of Chapter 58. Security: a physical device can be stolen or tampered with, so protect any stored data and secrets. Many real systems combine both approaches. A small local model handles quick, private, urgent decisions, and it passes harder tasks to a larger cloud model when a connection is available.