HOW LLMs WORK FOR THE NON-SPECIALIST

CHAPTER SUMMARY An LLM is a program trained on huge amounts of text that continues text by predicting one token at a time. Tokens are the chunks of text that models read and write, and cost, speed, and limits are all counted in tokens. The context window is the model's desk, and anything off the desk is invisible to it. Models are stateless, so any memory must be supplied by the harness, and temperature trades consistency against variety. LLMs are strong at language tasks but can hallucinate, which is why harnesses add retrieval, citations, and verification. KEY TERMS REVIEW LLM: a program trained on large amounts of text to continue and produce text. Parameter: an adjustable internal number set during training. Training: the process of tuning the parameters by guessing hidden text and correcting errors. Token: a small chunk of text that a model reads or writes. Tokenizer: the program that splits text into tokens. Context window: the limited number of tokens a model can consider at once. Stateless: keeping no memory between separate runs. Temperature: a setting that controls how random the model's choices are. Hallucination: a confident statement by a model that is false.