Week 1 checkpoint · v0.1
Keys, once you've clicked the player: Space play or pause · ← → 5 s · J L 10 s · C captions · < > speed · M mute · F full screen
Read the transcript
0:00Tested isn't tagged
Four criteria, and each one is waiting for a tick. You built all of this during the week, so you could tick all four right now, in about four seconds. But look at what those ticks would stand on: your memory of what's in the repo. Yesterday's recording shows why that matters. Everything looked finished, and then a fresh clone said: no module named llm.client. So today, nothing gets ticked from memory. Each criterion is a claim about your repo, and each claim gets a check that could say no.
By the end, three things. You can prove each criterion with a command, run on the commit your tag names. You can explain the week's four ideas out loud, without notes. And when a check finds a problem, you can fix it and bring the tag along. Fast lane: if you've already run these checks in a fresh clone of your tag, skip to chapter three.
Start with what counts for the checkpoint. It isn't your working folder, and it isn't the newest commit on main. It's the one commit your v0.1 tag names. So where did yesterday's fresh clone run? In the recording, one commit earlier. Then the README, the changelog and the learning log went in, and the tag went on top. So the commit that was tested isn't the commit that was tagged. Three text files: probably fine. But 'probably' is where yesterday started, so test the tag itself.
So make a new scratch folder and clone again. It's the same test as yesterday, with one question added. Ask git where this folder stands, measured from the nearest tag: git describe, with the tags option. It prints v0.1, and nothing after it. So this folder holds exactly the tagged commit. A number and a hash after it would mean main has moved past the tag: that's chapter four. And 'no names found' would mean the tag never reached GitHub.
Criterion one allows you two things. The first is uv sync: twenty-three packages, straight from the lock file. The second is one environment variable: your key, set in this terminal and written to no file. If the clone needs a third thing, like a file copied over, the criterion isn't met yet.
Now run the warm-up, as a module. Three replies come back, each under its status line. Then a last line: bad request, four hundred. That's Day six's rejected temperature setting, caught by name. It's supposed to be there, and it's about to be evidence. And look at the first reply: eighty-nine tokens. Yesterday, the same command in a fresh clone gave eighty-two. You know why by now, and in chapter three you'll say it out loud. So the tag runs, from nothing but uv sync and a key.
3:21A check that could say no
The other half of criterion one says no key is in the repo. How do you prove that something isn't there? You search, in two places. First, the files git tracks. One has env in its name, and it's the example file, which holds a placeholder. Second, the history. A key that was committed and later deleted is still in the earlier commit. So search every change ever committed for the characters each key starts with, and count the lines: zero. Criterion two has the same shape: no call may go around the log. So search the code for the API call itself. One hit, inside the function that writes the log row.
That run just wrote a log in this clone, and it has three lines. But the run sent four requests. So is a row missing?
No. The fourth request is the one the API rejected. Nothing came back, so there was no stop reason and no token count to write down. Open the first row and read it against the criterion: model, stop reason, input tokens, output tokens, latency. Five fields asked for, five present.
Criterion three has two halves, and you've seen the first. That last line is the unsupported parameter: one readable sentence, no traceback. The second half is harder to see: where does the model ID come from?
So test it. Set LLM_MODEL to a name no model has, and run again. Which result means you pass: three normal replies, one clear line that names the model, or a long traceback?
One clear line. Three normal replies would mean a model ID is typed into the code, and your setting was ignored.
And here it is: model not found, four oh four, LLM_MODEL is no-such-model. For once, the error is the pass: the code used the name it was given.
Criterion four: EXPERIMENTS.md records four or more runs. Open the file as the tag holds it, and count the rows with numbers in them. The first pass alone has seven. What wouldn't count is a heading, and a plan to fill it in later.
Last, the starter's precheck, pointed at the tag. Four lines say ok. But it only lists file names and never opens a file, so an empty EXPERIMENTS.md would pass it too.
Criterion one: a fresh clone of the tag ran on uv sync and one variable, and two searches found no key. Two: one place calls the API, three replies made three rows, and the first row has all five fields. Three: an unsupported setting got one clear line, and a made-up model name got an error that names it. Four: seven runs with numbers, on a commit git describes as v0.1. Four ticks, and each rests on something that could have said no.
7:02Say it without notes
Four ticks say what your repo can do. The checkpoint's last line is about you: four things to explain without notes. So say each answer out loud, in the pause, before the narration does.
First: twenty identical requests went in, and six different answers came out. Why does the same prompt give different answers?
The model doesn't pick a token. It scores every possible next token, and the sampler turns those scores into odds and draws one at random. Each request makes its own draws, so each can go a different way. A thin answer stops at 'it's random'. So where is the randomness? Not in the model's scores, which were the same all twenty times, but in the draw that comes after.
Second: one conversation, three turns, and the input went from twenty tokens to a hundred and twenty-two, then a hundred and ninety. Why do long chats cost more per turn?
Because the API keeps nothing between requests. Your code sends the whole conversation again on every turn, and you pay for all of it as input. A thin answer says the chat got longer. A full one has a number in it: of three hundred and thirty-two input tokens here, a hundred and forty-two had been sent before.
Third: this reply came back with stop reason max_tokens. What does that tell you, and what do you do about it?
The cap ran out: forty tokens allowed, forty used, and the model was still writing. So treat the text as incomplete, however finished it looks, then raise the cap or ask for less. A thin answer says the prompt was too long. But the cap is on output. And it can run out before the answer starts: this reply spent all forty tokens thinking, and came back with no text.
Fourth: Day three's toy model, asked for the capital of France four times, gave four fluent answers, and none of them is Paris. Why can a model be confidently wrong?
Because it was trained to continue text plausibly, and nothing in that procedure checks a fact. Right and wrong answers come out the same way, in the same even tone. A thin answer blames bad data. But even a correct answer was never checked: it was only likely. That's why the fixes come from outside the model: grounding in Week three, and evals in Week five. If one of the four came out thin, the label above its picture names the day to rewatch.
10:23Before you tick
Suppose a check had said no: say, a file never reached the clone. You fix it in your working folder and commit. This recording's repo passed, so its commit is a stand-in: one line in the learning log. Then you push. So is the checkpoint satisfied now?
Ask git. It prints the tag, then a one, then a hash: main is one commit past v0.1. The fix is on GitHub, and the tag still names the repo from before it.
Yesterday showed two commands for this. The first is git tag with the force flag: it moves v0.1 to the commit you're on. The second pushes the tag, again with the force flag, because GitHub already has a v0.1 and won't replace it otherwise. 'Forced update' is the line you want. Ask again: v0.1, and nothing after it. On your own repo, this week, moving a tag is fine. On a shared project, others may already have the old one, so make a new tag instead.
One trap is left. The clone you just tested in can't check this: even after a pull, it keeps the tag it first saw. So clone again, into a new folder. v0.1. Now run the same list here, from uv sync down: it's the only run that tests the commit that counts.
Here's why. Today's checks ran on this commit, and the tag has moved one past it. Run the list in the new clone, and the two stand together: the commit you tested is the commit you tagged.
Now open the checkpoint page and tick the four criteria, each with a result you can point to. The page also has a short knowledge check on the week. Take it before you rewatch anything, so it measures what you know, and not what you just heard.
And you're ready to move on when you can explain these four without notes, as fully as chapter three did.
Self-check one. Which of these is evidence that no key is in the repo: you remember adding .env to .gitignore; a history search for the key's prefix prints zero; or the script runs without an error?
The history search. It's the only one that could have said no: memory is what today replaced, and a script runs just as well with a key committed.
Self-check two. The precheck says all path checks pass. Which of the four criteria has it proved: all four; the fourth, four or more runs recorded; or none in full, because it only checks that files exist?
None in full. It lists file names at the tag and never opens a file, so an empty EXPERIMENTS.md passes it.
Self-check three. You fix a bug, commit and push, and git describe prints the tag, a one, and a hash. What does the tag hold: the repo before the fix; the fix, because the push moved the tag; or nothing, because the push removed it?
The repo before the fix. A tag names one commit and stays there until you move it.
That's the Week one checkpoint: four criteria with evidence, and four explanations in your own words. Yesterday, one change moved the numbers on both passes without cutting the answer short: one sentence of system prompt. So the words you send can matter more than the settings around them, and that's where Week two starts. Day eight: prompt basics. This lesson was narrated with an AI voice.
your first logged calls
Move the client and logger into llm/. Vary max_tokens, the system prompt, and effort and temperature where supported. Responses can hold several blocks (thinking, text), so keep only the text; at 40 tokens it may be cut off or empty. Try temperature only where supported: current Python SDKs no longer take it as a keyword, so pass extra_body={"temperature": 0.2}, and the latest Claude models return a 400 error for it, so catch anthropic.BadRequestError. On OpenAI, use responses.create(...) and resp.output_text.
import os
import anthropic
MODEL = os.getenv("LLM_MODEL", "claude-haiku-5-5") # set to a current model ID
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY from the environment
def ask(prompt: str, max_tokens: int = 400, **options) -> str:
resp = client.messages.create(
model=MODEL, max_tokens=max_tokens,
messages=[{"role": "user", "content": prompt}], **options)
u = resp.usage
print(f"[{resp.stop_reason}] in={u.input_tokens} out={u.output_tokens}")
return "".join(b.text for b in resp.content if b.type == "text")
q = "In two sentences, what is a context window?"
print(ask(q))
print(ask(q, max_tokens=40)) # watch stop_reason
print(ask(q, output_config={"effort": "low"})) # effort, where supportedCreate a free account to tick off the criteria, take the knowledge check and keep your progress.
Start free