How models are built
Deep Dive into LLMs, first hour.
Finish the Deep Dive.
Keys, once you've clicked the player: Space play or pause · ← → 5 s · J L 10 s · C captions · < > speed · M mute · F full screen
Read the transcript
0:00A model you can read
This is a language model, built for this lesson out of the course's ten guides. Building it took under half a second. Give it the start of a sentence, and it writes the rest. It sounds a little like the guides, and it means nothing. So ask it for a fact. The capital of France is: a database, and deletes when sources change. Wrong, of course. But look at how it's wrong: a complete sentence, clean grammar, no hesitation, and no sign that it doesn't know. The models you'll call this week are enormously bigger, and they can still do exactly this. So today you take the small one apart, because it fails for the same reason, and here you can read every part.
By the end of today, three things. You can say what a model predicts at each step, and why the same prompt can give different answers. You can explain why a model can be confidently wrong, and name the three fixes this course uses. And you can say how AI engineering differs from machine learning, which tells you what your job is, and what it isn't. Fast lane: if you can already explain why a model with its randomness turned off can still be confidently wrong, skip to the last chapter, then go straight to today's video.
Today's task is one hour of video: the first hour of the Deep Dive, linked on today's page, with LEARNING.md open beside it.
It's a dense hour. So before you press play, build the smallest model that could possibly work. Take the ten guides: a hundred and ninety thousand words. Cut them into tokens, exactly as you did yesterday, and you have two hundred and sixty-six thousand of them. Then count. For every pair of tokens in a row, write down what came next, and how often. That table is the entire model: a hundred and twelve thousand rows of counts. A real model has the same outline: text, tokens, then a step that boils the text down into numbers. But that step is training, not counting, and by chapter three you'll see why counting can't do the job.
2:32One token at a time
Here's the whole trick, slowly. The text so far ends with 'the model'. So ask the pile of text one question: every time those two tokens appeared, what came next? They appeared two hundred and ninety-five times, and a hundred and thirteen different tokens followed. Apostrophe s came next twenty-five times, 'can' seventeen times, a full stop fourteen. Divide each count by the total, and the counts become odds: eight and a half percent, five point eight, four point seven. That's one of today's answers. At each step, a model predicts odds for every token that could come next, and nothing more. To write anything, something has to pick. So draw a random number between zero and one: point three three two. Lay the odds end to end and see where it lands: on 'sees', which had under three percent. Append it, slide the window along by one token, and ask again. 'Model sees' appeared eight times, with five different continuations. This time the draw is point nine six, which lands at the far end: on 'numbers', a token that followed just once. Keep going. 'Sees numbers' appeared exactly once in the whole text, so there's a single candidate, and the draw can't change anything. Same for the next token, and the one after. That loop is how a language model writes: read the context, give odds for the next token, pick one, append it, repeat. The big models compute their odds very differently, but they run this same loop, one token at a time.
So predict. Nothing in the table changes between runs: same guides, same counts, same prompt. Run it a second time: do you get the same text, or different text?
Different text. The model didn't change at all; the odds at the first step are identical. What changed is the random number.
Here's the run you just watched. Its first draw was point three three two, and that put it on 'sees'. Twice more, same prompt. A draw of point seven five lands on 'writes'; point four lands on 'answers'. After that first token, each run is in a different part of the table, and the texts never meet again. This is called sampling, and it answers another of today's questions: why can the same prompt give different answers? Because the model only gives odds, and a random draw picks from them at every single token.
Notice where the draw sits: outside the model. So you can change how the picking is done without touching the model at all. That's what the settings called temperature and top-p do, and they're Day five.
5:51Why counting isn't enough
The toy's sentences fall apart quickly, and there's a tempting fix: let it look further back. To see the effect, start lower, with one token of context: 'the model can't talk to the model, and new requests without changes safely'. Two tokens is the version you've been watching. Phrases hold together now, but the sentence still wanders off. Now three tokens, and four. Read those two: 'stops and returns a confident answer'; 'search only what the signed-in user may read, and prove it with tests'. Those are real sentences. So more context gives better text. Why stop at four; why not ten tokens, or a thousand? Hold that thought, and try one experiment first.
Take the four-token version and run the same prompt two hundred times, with fresh random draws each time. With two tokens of context, that gives two hundred different texts. With four, how many: two hundred, about a hundred, or one?
One. Two hundred runs, two hundred sets of random numbers, and the first twenty tokens came out identical every time. So where did the randomness go?
Here are those two good sentences again. They sounded like the guides, and there's a plain reason for that. They are the guides. Everything underlined is copied word for word: the four-token run from Guide four, the three-token run from Guide seven. The model stopped writing and started reciting. And that's where the randomness went. With a long context, most rows in the table have exactly one continuation, so the draw has nothing to choose between.
Measure it properly: two hundred runs at each context length, counting any stretch of eight or more tokens that matches the guides word for word. With one token of context, almost nothing is copied. With two, about a fifth. With three, eighty-five percent. With four, almost ninety-nine. And the small print under each bar explains it: the share of rows with only one candidate climbs from forty-one percent to ninety-seven. So counting forces a trade. A short context gives you original nonsense; a long one gives you a perfect copy. If you've built a cache, you know this behavior: it only hits on an exact match, and a miss returns nothing useful.
Could more text fix that? Do the arithmetic. The guides use about fourteen thousand different tokens, so the number of possible two-token contexts is fourteen thousand times fourteen thousand: two hundred million. The guides contain a hundred and twelve thousand of them. That's about one in eighteen hundred, and this is the easy case, with only two tokens. Every extra token of context multiplies the possibilities by fourteen thousand again, and a real model reads thousands of tokens of context. No pile of text can fill that table. So a real model can't be a table of counts. It needs a way to say something sensible on a miss.
9:39Learned, not counted
Here's another way to fill in that row: same context, 'the model', same hundred and thirteen candidates. But this time, don't count. Start from knowing nothing, with every candidate at equal odds, a bit under one percent. Now compare those odds with what really came next, across all two hundred and ninety-five examples in the text. Apostrophe s deserved far more than one percent, so nudge its number up a little, and nudge the overrated ones down. That's one pass. After ten passes, the bars begin to separate. The white marks are the odds you got from counting, and the bars are creeping toward them. Fifty passes: apostrophe s is up to seven percent. Nobody gave it that number. It's just a little less wrong on every pass. Five hundred passes, and every bar sits on its white mark. Learning arrived exactly where counting did, without counting anything. The number on the right is called the loss: how surprised the model is, on average, by the token that really came next. Training means one thing: adjust the numbers so the loss goes down. Both words come up constantly in the video. And notice the loss didn't reach zero; it stopped near four point two. It can't go lower, because the text itself is uncertain here: a hundred and thirteen different tokens really did follow 'the model'.
So if learning lands where counting did, why bother? Because of where the numbers live. In the table, every context owns a private row, and learning one row tells you nothing about any other. A neural network has no rows. Every context runs through one shared set of numbers, the parameters, so a nudge for one sentence shifts the odds for similar sentences too, including ones it has never seen. That's a sensible answer on a cache miss, and it's why models are trained, not counted.
What comes out of training is easy to misread, and the toy can show you. This is its three-token version. Ask it a question: what is the capital of France? 'How much data, growing how fast?' It answered your question with a question, lifted from Guide six. Nothing in there is trying to answer you; it's continuing a document that happens to begin with your words. A model fresh out of training behaves the same way, at a far higher level of quality. It's called a base model: it continues text, and it doesn't reliably follow instructions.
So here's a loose map to hold while you watch. The video has the detail; your part is to pause at each new stage, and write one line on it in your own words. It begins with text, a very large amount of it. Listen for where that text comes from, because it's the first question you answer in your log. Then tokens, which you measured yesterday. Then the loop you just watched: predict the next token, lower the loss, over all of that text. This stage is called pretraining, and what comes out is the base model. Then a second stage. Keep training, but on a different kind of text: instructions paired with good responses. Same loop, new data, and the text-continuer becomes an assistant that answers.
Which sets up a mistake that's easy to make. You correct a model in the middle of a chat, it apologizes, and the next answer is right. It feels like you taught it something. Rewind, and ask what actually changed. Training finished long before your request arrived. Did a parameter move, or did you add tokens to the context? You added tokens. Your correction is in the transcript, and as you saw yesterday, the whole transcript goes back in with every request. The model rereads your correction each time; it never learned it. Open a new chat and it's the same model as before, with the same gaps. Rule of thumb: training changes the parameters; a request only changes the context. And what a model does at a gap is the last thing to see today.
14:29Confidently wrong
Back to the question from the very start. The text so far ends with 'France is', so run the same procedure: how often did those two tokens appear together in the guides? Never: the guides mention the capital of France exactly once, inside a question about caching, and nobody answers it. So there's no row, and the toy does the only thing it can. It drops a token, and asks what follows 'is' on its own. And 'is' appears eighteen hundred times, with five hundred and forty-five different continuations: 'a', 'the', 'an', 'about'. Every one of them is good grammar. Not one of them is a row that says: no data. So it draws. Point zero five nine lands on 'a', the most common choice, and the sentence is under way. From 'is a', there are a hundred and sixty-five possible continuations, and the draw lands on 'database'. After that it just follows the text. Now look at what never happened. At no step did anything check whether the sentence was true. The procedure has one output, odds for the next token, and it gives that output whether it has good data, thin data, or none.
Run it four times, and you get four capitals of France: a database, a subsidiary of Globex, 'correct', and 'out of scope'. Each one is fluent, and each one arrives in the same even tone. Now, a real model gets this question right. But not because it checked. In its training text, Paris follows those words so often that Paris takes nearly all the odds: right answers and wrong answers come out of the same procedure. So ask it about something its training text never covered, like your company's travel policy. Now it's in the toy's position: no data, no error, and a fluent answer anyway. That's a hallucination, and the course guide gives the cause in one line: models are trained to sound plausible, not to check facts.
Here's a natural idea: the variety is the problem, so remove it. No random draw; always take the most likely token. Predict what the toy says about France now: the right answer, that it doesn't know, or something still wrong?
Still wrong. Picking the top candidate changes which wrong sentence you get, and nothing else.
Here it is: the capital of France is a good default, because it needs calibrating against human labels. Run it again, and again: the same sentence every time. You've made it consistent. You haven't made it right, because the fact was never in the odds to begin with. Remember this on Day five, when you meet the setting that does this. Turning the randomness down makes answers more repeatable. It doesn't make them true, and it won't make a model admit a gap.
Which brings you to the mistake that matters most this week. An answer comes back well written and completely sure of itself, so it goes into the product unchecked. Rewind. You've now watched where that tone comes from: fluent is what the procedure produces every time, with data or without. The tone is not evidence. So the fixes come from outside the model, and this course uses three. Grounding: put the source text into the context, so the answer is read from it and not recalled. That's Week three. Abstaining: let the model say that the answer isn't in its sources. And evals: measure how often it's wrong, so you find out before your users do. That's Week five.
You can try grounding on the toy right now. Here's the plain prompt again, with its wrong answer. Now put the answer in the prompt, right in front: Paris is the capital of France. Same random draws as before. The capital of France is: a database, and deletes when sources change. Word for word the same. The answer was sitting in its own prompt and it couldn't use it, because this model only ever looks at its last two tokens, and both times those were 'France is'. So grounding needs something the toy doesn't have: a way to read the whole context, and pull out the one token that matters from thousands. That mechanism is called attention, and it's tomorrow.
19:41Your side of the line
You've spent this whole lesson on how models get built. So here's a fair question before you watch an hour more of it: is building them your job? No. Gathering the text, training, fine-tuning: that's machine learning engineering. Everything in that lane changes the parameters, and it ends with a pretrained model. Your lane starts where that one ends. You take the pretrained model as a given, the way you'd import a library, and build the product around it: with prompts, retrieval, tools and evals. Look at what those four have in common. None of them touches a parameter; they shape the context going in, or check what comes out. That's the paragraph today's task asks for: how AI engineering differs from ML, in your own words.
One trap before you press play. The video is three and a half hours long, and every stage in it opens a rabbit hole. It's interesting enough to lose a week in. Rewind to the task. It's the first hour, and the goal is a mental model, not the mathematics of training. You build on pretrained models. So the test is simple: can you explain how a model works, out loud, without notes? Once you can, stop, and move on.
Your turn. Open LEARNING.md beside the Deep Dive video, linked on today's page, and watch the first hour. Pause at each new stage of how a model is built, and write one line on it in your own words. Then answer three questions in the log: where the training text comes from, what the model predicts at each step, and why the same prompt can give different answers. A toy has shown you two of those; now say them for the real thing. Add two sentences on why a model can be confidently wrong, and name the fixes: grounding, abstaining and evals. Then finish with your paragraph on AI engineering and ML. And if you're on the student track, there's one extra today: finish the rest of the Deep Dive.
On the map, today is one file again. LEARNING.md gets today's answers under Week one; commit it once they're written.
Done when, exactly as the site puts it. LEARNING.md answers, in your own words, what the model predicts at each step, and why the same prompt can give different answers. Why it can be confidently wrong, and how AI engineering differs from ML. Four answers in your own words, and Day three is done.
Self-check one. You send one prompt five times and get five different answers. Is that because each next token is drawn from the model's odds, because the model learned from your earlier requests, or because the provider sends you to a different model each time?
Each next token is drawn from the model's odds. The model gives the same odds every time; the draw is what differs, and one different token early on changes everything after it.
Self-check two. A model never saw your company's travel policy, and you ask it about that policy. What's most likely: an error because the data is missing, a plain 'don't know' if the randomness is turned off, or a fluent answer that may be wrong?
A fluent answer that may be wrong. It's trained to sound plausible, not to check facts, and always picking the top token doesn't put the missing fact into the odds.
Self-check three. Your assistant gives outdated answers about your product. Which fix is AI engineering: retrain the model on your documentation, put the current docs into the context, or correct it in a chat and wait?
Put the current docs into the context. Missing knowledge is a retrieval problem, and retrieval works on the context while the parameters stay exactly as they are.
That's Day three. Tomorrow is Day four, transformers: how a real model reads its whole context and not just the last two tokens, with a mechanism called attention. This lesson was narrated with an AI voice.
How to do it
- 1Watch the first hour with
LEARNING.mdopen. Pause at each new stage of how a model is built and write one line on it in your own words. - 2Answer three questions in the log: where the training text comes from, what the model predicts at each step, and why the same prompt can give different answers.
- 3Explain in two sentences why a model can be confidently wrong, then name the fixes this course uses: grounding (Week 3), abstaining, and evals (Week 5).
- 4Write one paragraph on how AI engineering differs from ML: you build products on top of pretrained models, with prompts, retrieval, tools and evals, rather than training them.
LEARNING.md answers, in your own words, what the model predicts at each step, why the same prompt can give different answers, why it can be confidently wrong, and how AI engineering differs from ML.
Going deep on training internals. You will build on pretrained models, so stop once you can explain the mental model without notes and move on.
Create a free account to mark steps done and keep your progress. Week 1 is free.
Start free