FROM MODEL TO APPLICATION

CHAPTER SUMMARY An API is a doorway through which software sends requests to a model and receives responses, usually over HTTPS with JSON data and an API key. Inference is running a trained model to get an answer, and it is where cost and delay occur. Modern models take a list of role-labeled messages, and because models are stateless the whole list is resent each time. Open-weight models give control but demand hardware and skill, while closed hosted models give convenience but create dependence on a provider. GPUs do many calculations at once, which is why models need them. KEY TERMS REVIEW API: rules that let one program request work from another. Endpoint: a specific web address that accepts a certain kind of request. API key: a secret string proving who is making a request. JSON: a plain-text format of labeled data. SDK: a code library that simplifies using a service. Inference: running a trained model to get an answer. Prompt: the input text given to a model. Completion: the text a model returns. Role: a label saying who is speaking in a message. Open-weight model: a model whose trained weights are published for download. Closed, hosted model: a model reachable only through its provider's API. GPU: a chip that performs many calculations in parallel.