HOW LLMs WORK FOR THE NON-SPECIALIST

CHECK YOUR UNDERSTANDING • Why can one word be more than one token? ◦ Answer: Because the tokenizer splits text into pieces, and long or rare words are often divided into several pieces. • What happens to text that does not fit inside the context window? ◦ Answer: The model cannot see it, so the harness must shrink, summarize, or leave it out. • Chat applications seem to remember you, so how is the model stateless? ◦ Answer: The application resends the entire conversation with each new message, so the model rereads rather than remembers. • Which temperature would you choose for extracting a date from an invoice, and why? ◦ Answer: A temperature near zero, because you want consistent and predictable output. • Why can you not trust a model simply because it sounds confident? ◦ Answer: Because models produce plausible text whether or not it is true, so confident wording is not evidence of accuracy.