DOMAIN-SPECIFIC HARNESSES

DATA ANALYSIS AND BUSINESS-INTELLIGENCE AGENTS Business intelligence (BI) means using data to understand how a business is doing and to help make decisions. A data analysis agent answers questions about data, such as why did sales drop in the Northeast region last quarter. To do this it may query a database, run its own exploratory calculations, make charts, and write an explanation backed by specific numbers. Exploratory analysis means poking at the data from several angles to see what stands out. A query is a question written in a database language, such as SQL, that asks the database to return certain information. These agents face particular risks. Wrong queries: a plausible-looking query may count the wrong thing, double-count rows, or use the wrong time period, and give a confident but wrong answer. Misleading stories: finding a pattern and inventing a cause for it, when the data cannot support the cause, is a classic error. Data access: the agent may see sensitive data, such as salaries or customer records, so it needs least-privilege access, as in Part Nine, preferably read-only. Destructive commands: a database tool should usually forbid commands that change or delete data. Cost: a careless query over a huge table can be slow and expensive. Good harnesses reduce these risks by showing the actual query alongside every answer so that an analyst can check it, by running on read-only copies of data, by limiting the size and cost of queries, and by testing the agent on questions whose correct answers are known. They also encourage the agent to say when the data cannot answer a question.