HOW LLMs WORK FOR THE NON-SPECIALIST

STRENGTHS AND WEAKNESSES LLMs have real strengths. They are strong at writing and rewriting text, summarizing, translating, explaining, classifying, extracting facts from messy documents, writing and explaining code, and following instructions in ordinary language. They are flexible, since the same model can handle a recipe, a contract, and a spreadsheet formula. They are also fast and available at any hour. They also have real weaknesses. They can make arithmetic and logic mistakes, especially in long multi-step problems. They depend on whatever is in the context window and know nothing of events after their training ended unless they are given fresh information. They can be thrown off by misleading instructions hidden inside documents, a risk you will study in Part Nine. They can be inconsistent, giving different answers to the same question on different tries. And they can hallucinate. Hallucination is when a model states something false with confidence. The word is a metaphor. The model is not seeing things; it is producing text that fits the pattern of a correct answer even though the content is invented. A famous example type is a fake citation, where a model asked for a legal case supports its answer with a case name, a court, and a date that all sound real but do not exist. Hallucination happens because the model's job is to produce plausible continuations, not to check facts. If it does not know something, the pattern of a confident answer is still very strong, so it may produce one anyway. The practical lesson is that you cannot rely on how sure the model sounds. Instead, a harness gives the model ways to look things up, requires sources, and checks claims. Part Eight, the verification layer, is dedicated to those protections. Each weakness above maps to a harness feature. Forgetting maps to memory management, mistakes map to verification, risky actions map to permissions, and cost and delay map to budgets. Seeing this mapping early will make the rest of the book feel logical.