Large Language Models (LLMs): How ChatGPT Really Works

ChatGPT can write essays, debug code and explain quantum physics — so it's easy to imagine something magical inside. But in Lesson 19 you met the real engine: the Transformer. A Large Language Model (LLM) is simply a very big Transformer trained on a huge amount of text to do one deceptively simple thing: predict the next word. This lesson opens the lid and shows you exactly how that one trick becomes something that feels like intelligence.

What makes it "large"

The "large" is literal. Modern LLMs have billions of parameters (the weights you learned about in Lesson 16) and are trained on a big slice of the public internet, books and code. More parameters + more data + more compute has, so far, kept producing more capable models. But the mechanism never changes — it's next-word prediction, scaled up.

Step 1: Text becomes tokens

An LLM doesn't read letters or whole words — it reads tokens, which are common chunks of text (a short word, part of a longer word, or punctuation). Breaking text into tokens is called tokenization.

This is why you sometimes hear limits described in tokens, and why odd spellings can confuse a model — it never saw the letters, only the chunks. A rough rule of thumb: 1 token ≈ 4 characters ≈ ¾ of a word in English.

Step 2: Predict the next token

Here's the whole secret. You give the model some tokens (your prompt), and it outputs a probability for every possible next token in its vocabulary. It picks one, adds it to the text, and repeats — token by token — until the answer is complete. That loop, running on a massive Transformer, is all of ChatGPT.

Temperature: creativity vs. focus (runnable)

How "adventurously" the model picks from those probabilities is controlled by a setting called temperature. Low temperature = play it safe (always the top choice, more predictable). High temperature = take risks (more variety and creativity, more mistakes). This NumPy snippet shows the exact same scores becoming sharper or flatter as temperature changes.

import numpy as np

tokens = ["blue", "clear", "grey", "falling"]
logits = np.array([3.0, 1.2, 0.7, -0.5])   # model's raw scores

def probs(logits, temperature):
    z = logits / temperature
    e = np.exp(z - z.max())
    return e / e.sum()

for t in [0.2, 1.0, 2.0]:
    p = probs(logits, t)
    print(f"temp={t}:", dict(zip(tokens, np.round(p, 2))))

# temp=0.2 -> almost always "blue" (focused)
# temp=2.0 -> probabilities flatten (creative, riskier)

When you use ChatGPT's API, temperature is a dial you control. Low for facts and code; higher for brainstorming and stories.

How an LLM learns to be helpful

Raw next-token prediction gives you a model that completes text, not one that helps you. Turning the first into the second takes three stages:

RLHF (Reinforcement Learning from Human Feedback) is the step that turned a text-completer into ChatGPT — the polish that makes it feel like it's actually talking to you.

The context window: its short-term memory

An LLM has no memory between separate chats. Within one conversation, everything it can "see" — your messages and its own replies — must fit inside its context window, measured in tokens. Modern models hold tens or hundreds of thousands of tokens. Go beyond that and the earliest parts scroll out of view, which is why a very long chat can make the model "forget" what you said at the start.

Why LLMs hallucinate

Because an LLM predicts plausible next tokens — not verified facts — it can state something false with complete confidence. This is called a hallucination. It's not lying; it's pattern-completion producing text that looks right. That's why you should always verify important facts, citations and numbers an LLM gives you. It's a brilliant assistant, not an oracle.

What real code looks like

Using an LLM in your own program is a few lines. Here's the shape of a typical call (pseudocode close to real SDKs; needs an API key and the provider's library):

# Conceptual shape of an LLM API call
response = client.chat.create(
    model="a-large-language-model",
    messages=[
        {"role": "system",  "content": "You are a helpful tutor."},
        {"role": "user",    "content": "Explain tokens in one line."},
    ],
    temperature=0.3,          # low = focused, factual
)
print(response.output_text)

The system message sets behaviour, the user message is your prompt, and temperature is the dial from earlier. Getting more out of that user message is a skill of its own — which is exactly the next lesson.

Keep this in mind: an LLM doesn't "know" facts the way a database does — it predicts likely text. That's why it's astonishingly fluent and capable of being confidently wrong.

Understanding "it's just predicting the next token" changed how I use these tools day to day — I stopped treating ChatGPT as an all-knowing answer machine and started treating it as a fast, fluent draft-writer I always fact-check. That single mental shift made my results better and my surprises fewer. Try holding that frame the next time you use it and see if it changes how you prompt.

Key takeaways

• An LLM is a giant Transformer trained to predict the next token; that one loop produces everything ChatGPT does.
• Models read tokens (chunks of text), not letters or whole words.
• Temperature controls randomness: low = focused/factual, high = creative/risky.
• Training goes pretraining → fine-tuning → RLHF; RLHF is what makes it helpful and aligned.
• The context window is its short-term memory, and hallucinations happen because it predicts plausible text, not verified facts — always check important claims.

Continue the series: ← Lesson 19: Transformers and Attention  ·  Next: Lesson 21 — Prompt Engineering: How to Get Great Answers from AI →