DEPLOYMENT ARCHITECTURES

HOSTING MODELS AND HARNESSES An agent system has two main parts that need a home. The first is the harness, the code that runs the loop, tools, memory, and rules. The second is the model, the large AI system that does the thinking. You can decide where each is hosted independently. For the model, there are three common choices. Hosted model API: a provider runs the model on its own computers and you call it over the internet through an API, which is a defined way for one program to ask another for services. You pay per use and do not manage the hardware. This is the easiest option and gives access to powerful models, but your data travels to the provider, and you depend on their availability and pricing. Self-hosted model: you run an open model on your own servers or rented machines. You control the data, and costs can be predictable at high volume, but you need expertise and expensive hardware, and the most capable models may not be available this way. Hybrid: some tasks use a hosted API for hard problems and a self-hosted smaller model for simple or sensitive ones. For the harness, the choices include running on your own servers or virtual machines, on a managed cloud platform, in containers, or on serverless functions that start only when needed. A container is a packaged bundle of software and everything it needs to run, which behaves the same on any machine. Serverless means the cloud provider manages the machines and you pay only when your code runs. Think of a restaurant. The harness is the kitchen staff and the model is the specialty supplier that provides a key ingredient. You can run your own kitchen and buy the ingredient from a supplier, or grow the ingredient yourself. Decide based on constraints such as budget, data residency (rules about which country data must stay in), privacy needs, expected volume, and your team's skills. Many agents also run work inside sandboxes, so your architecture must include where those sandboxes live and how they are created and destroyed.