HOW LLMs WORK FOR THE NON-SPECIALIST

EXAMPLES Example 1: Splitting a word into tokens. Consider the word "unbelievable." A tokenizer might split it into three tokens, such as "un", "believ", and "able." Walk through what that means. The model does not see one word; it sees three pieces, each with its own number. So token count is not the same as word count. If you pay per token, a short sentence full of rare or long words can cost more than a longer sentence of simple words. Exact splits differ by model, so treat this as an illustration rather than a rule. Example 2: How much fits in a context window. Suppose a model has a context window of two hundred thousand tokens. That is roughly equal to a five hundred page book. Imagine you attach a long report that uses one hundred and fifty thousand tokens, then add your instructions and a conversation history of thirty thousand tokens. You have used one hundred and eighty thousand tokens and have only twenty thousand left for the answer and anything else. If you now try to attach another large document, it will not fit. Anything outside the window is invisible to the model unless it is supplied again. A harness must therefore track how full the window is and make choices before it overflows. Example 3: Two temperature settings. A user asks, "What is the boiling point of water at sea level in degrees Celsius?" With temperature at zero, the model replies one hundred degrees every time, which is what you want. Now a different user asks for ten names for a new bakery. With temperature at zero the ten names will come out very similar and repeat on every try. With temperature around 0.9 the names are livelier and different each time. The lesson is to match the setting to the job, using low values for facts and structured output, and higher values for creative work. Example 4: A fake legal citation. An assistant is asked to find a court case supporting a certain argument. It answers with a confident name, such as a plausible-sounding pair of companies, a court, and a year. A lawyer who trusts it discovers later that the case does not exist. This is a hallucination. In a harness, you would prevent the harm by adding a verification step: after the model proposes a citation, a tool looks up the case in a trusted legal database, and the answer is only accepted if the case is found. Part Eight covers how to design such checks.