DEPLOYMENT ARCHITECTURES

EXAMPLES Example 1: Hosting A company builds a contract-summary agent. It runs the harness on its own cloud servers, where it can reach internal documents safely. For thinking, it calls a model provider's hosted API. Two independent decisions are therefore made. The harness is hosted by the company, giving control of tools, logging, and permissions. The model is hosted by the provider, giving access to a strong model without buying hardware. Later, a legal team asks that certain highly confidential documents never leave the company network. The company adds a self-hosted smaller model for those documents only, creating a hybrid design, and changes nothing about the harness logic besides routing. Example 2: Scaling A harness is first designed for one analyst using one session at a time. It stores conversation state in the computer's memory and runs one job at a time. When the company wants 10,000 simultaneous sessions, several problems appear. Memory-based state is lost if a machine restarts, and sessions cannot move between machines. One slow session blocks others waiting behind it. The team moves state into a shared database, adds a queue, runs many identical workers behind it, uses connection pooling for the database, sets a per-session time limit, and divides the provider rate limit among users. Now a stuck session affects only itself, and adding machines increases capacity. Example 3: Configuration and secrets A team keeps three configuration files: one each for development, staging, and production. The development file points to a fake customer database and a test API key with a low spending cap. The staging file points to a realistic copy of the data with scrubbed personal details. The production file points to the real database, and its keys live in the secrets manager, available only to the production system. One day a developer accidentally runs a test script that deletes records. Because their machine has only the development configuration, the script wipes fake data and nothing else. No real customer is affected. Example 4: Gradual rollout A team has improved their agent's main prompt and wants to release it. Stage one: 1 percent of traffic for one day. Stage two: 10 percent for one day. Stage three: 50 percent for one day. Stage four: 100 percent. At each stage, the team monitors error rate, cost per run, user ratings, and the number of escalations. An automatic rollback rule says that if the error rate exceeds twice the baseline or cost per run rises more than 30 percent, traffic switches back to the old version. At the 10 percent stage, cost per run rises 45 percent because the new prompt triggers extra tool calls. The rollback fires automatically, only 10 percent of users were affected, and engineers fix the cause before trying again.