Key takeaways
“Large language model” has shown up in nearly every business deck of the past three years. Almost none of them defined it. Here’s a working definition. It’s no research paper, but it’s enough to read the news and judge a product’s claims.
One step, repeated
Strip everything else away and a language model is a function. You give it text. It returns a list of possible next tokens, each with a probability.
Ask it What is the capital of France? It turns your question into tokens and runs them through a network with billions of parameters. It gets back something like Paris at 0.92, Lyon at 0.03 and the at 0.02, with a long tail behind them. It picks one, usually the likeliest, sometimes with a little randomness. It adds that to the input and goes round again for the next one.
That loop is the whole engine. Run it with a system prompt that says “you are a helpful assistant” and you have ChatGPT. Run it with tools attached and you have an agent. What looks like answering, reasoning and writing comes from doing one prediction extremely well, billions of times in a row.
Repeat one operation (predict the next token) extraordinarily well, and you get text generation, ChatGPT, and agents alike.
Tokens aren’t words
A token is the unit a model was trained to read. It can be a word (apple), a piece of one (unpre, dictable), a character or a punctuation mark. Frontier models use byte-pair encoding. Common patterns get one token, rare ones get several.
In English a token is about three-quarters of a word. So a 1,000-word essay is about 1,300 tokens.
This matters more than it sounds:
Tokens are also the unit behind embeddings, the vectors models use to represent meaning.
What does attention do?
How a model predicts the next token is mostly about attention. Older designs forgot the start of a long paragraph by the time they reached the end. Attention fixes that. It lets every token in the input weigh in on the next one, based on how relevant it is.
Take “The boy who lost his hat looked sad. He was looking for his ___”. Attention is what lets the model reach back twenty tokens to hat.
The 2017 paper that introduced the modern transformer was called Attention Is All You Need. Today’s models stack dozens of these layers. Each works on the output of the one before and picks up more abstract patterns. Eventually it’s good enough to write working code.
Training and use are different
Training takes months. Text runs through the model, and its hundreds of billions of parameters get adjusted again and again to predict the next token better. It runs on thousands of GPUs and costs tens of millions of dollars.
Inference means using the trained model. The weights stay frozen. Each request costs a fraction of a cent, but that cost grows with every user.
So when someone says the model learned from their conversation, it usually didn’t. Products with “memory” save notes about you and add them to the prompt next time. The model itself doesn’t change.
How much can it read at once?
The context window is how much text a model can attend to in one request. Five years ago it was 2,000 tokens. Today it’s up to a million or more on frontier models.
Bigger isn’t automatically better. Every extra token costs money and time. And the middle of a long input is the hardest part for a model to use. I cover that in what a context window really is. For most work, fetching the few thousand tokens that matter beats sending everything.
What an LLM can’t do
It’s no search engine. It looks nothing up unless you give it a search tool.
It’s no database. It holds a rough statistical sense of its training data, not stored facts.
It’s no calculator. It copies arithmetic on familiar sums and fails unpredictably on unfamiliar ones.
Newer “reasoning” models do solve problems in several steps. But the mechanism is still next-token prediction, trained to write out a chain of thought before answering, as Anthropic’s developer docs describe.
An LLM by itself is a text predictor. Wrap it in tools, retrieval, and a control loop, and you get something that can act in the world.
Picture someone who has read most of the public internet. They only speak by finishing your sentences. They remember nothing between conversations. Sometimes they make things up, because they were trained to sound sure. That’s roughly who you’re talking to.
It explains why these models write well, stumble at maths, state errors with confidence and get better when allowed to work step by step. Agents, tool use and everything else run on the same trick: predict the next token, then do it again.
Frequently asked questions
What does LLM stand for?
Large Language Model. 'Large' is the number of parameters, usually hundreds of billions for frontier models. 'Language' is the domain. 'Model' is the maths: a function that turns input text into output text.
Is an LLM the same as ChatGPT?
No. ChatGPT is a product. The LLM (GPT-5.5, right now) is the model underneath. ChatGPT wraps it with a chat window, memory features, web browsing and other product layers.
Does an LLM think?
Not in any way humans would recognise. It runs forward passes through a neural network: billions of multiplications that give probabilities for each next token. The output can look like reasoning. Underneath, it's pattern completion at a very high level.
Why do LLMs hallucinate?
They predict text. They don't look up facts. When the training data doesn't cover a question (or covers it unclearly), the model still has to pick a next token. It picks the most statistically likely one. That answer might not be true.
How much does it cost to run an LLM?
On an API, you pay per token (usually $0.30-$15 per million tokens, depending on model and cache hit rate). If you host your own, you pay for the GPUs that hold the weights. A single frontier model can run on a single 8-GPU node.
Three-minute lesson on this What AI actually is Marketing course, basic. Free, no sign-up.