FROM MODEL TO APPLICATION

HARDWARE BASICS AND THE GRAPHICS PROCESSING UNIT Running a model involves an enormous number of simple calculations done at the same time. An ordinary computer chip, the Central Processing Unit (CPU), is a flexible worker that handles a few tasks very quickly one after another. A GPU is a chip with thousands of small workers that perform many similar calculations in parallel, meaning at the same moment. GPUs were first designed to draw video game graphics, which also require many parallel calculations, and it turned out they were ideal for neural networks. Think of the difference between one brilliant mathematician and a stadium of thousands of students who each do a small sum at the same time. For the huge, repetitive arithmetic inside a model, the stadium wins. GPUs matter for three reasons. First, they are expensive and sometimes scarce, so they drive the price of both training and inference. Second, a model must fit into the memory of the GPU, and larger models need more or bigger GPUs, which limits what you can run on your own machine. Third, the speed of the GPUs shapes how quickly a response arrives. If you use a hosted model, the provider handles the GPUs and you simply pay for usage. If you run an open-weight model yourself, you must buy or rent GPUs and manage them, which is a strong reason many teams begin with a hosted model and consider self-hosting later.