FROM MODEL TO APPLICATION

PROMPTS, COMPLETIONS, AND CHAT MESSAGE FORMATS A prompt is the text you give the model to work on. A completion is the text the model gives back. The words come from the early days when models were used mainly to complete a piece of text. Today most models are used in a chat style, where the input is a list of messages, each labeled with a role. A role says who is speaking. The most common roles are these. The system role carries standing instructions about how the model should behave, such as its job, tone, and rules. The user role carries what the person, or the program acting for the person, is asking. The assistant role carries the model's own earlier replies. Some APIs add a role or a message type for the results of tools, which is how an agent hears back about actions it took. A simple message list might read as follows. Message one has the role system and the content "You are a helpful assistant." Message two has the role user and the content "Summarize this report." The harness sends both, and the model returns a new assistant message containing the summary. Because models are stateless, as you learned in Chapter 2, a longer conversation is simply a longer list, with all earlier messages included again. Be aware that different providers use slightly different names and structures. One may call the standing instructions a system message, while another may place them in a separate field. This is one reason harnesses include a model adapter, a component you will meet in Chapter 8, that translates between a common internal format and each provider's format. Also be aware that APIs change over time. Always consult the provider's current documentation before relying on a field name or a model name.