PixioLab
Day 62.5–3 hWeekend

Build

hello_llm.py from the Claude quickstart or the OpenAI quickstart: calls, stop reasons, token counts, errors.

Student track adds

Same against a second provider or a local model with Ollama.

Read the transcript

0:00One call

This is where today ends. One file, written by you, makes three calls to a model. Before each answer it prints a line: why the model stopped, how many tokens went in, and how many came out. This is a real run, recorded for this lesson. Your words and your counts will be different, and yours are the ones that matter. Now look at the middle call. It was allowed forty tokens of output, it used all forty, and the answer stops halfway through a sentence. No error, and no warning: half an answer, printed exactly like a whole one. If a program were reading that line instead of you, how would it know? And in chapter four, another question under the same cap comes back with no text at all.

By the end of today, three things. You can call a model from your own code and read everything that comes back: the stop reason, the token counts, and only then the text. Every call you make writes one line to a log: which model answered, why it stopped, tokens in and out, and how long it took. And when a request fails, your script prints one line you can act on, instead of thirty lines of stack trace. Fast lane: if you've shipped code that checks the stop reason before it trusts the text, skip to chapter eight. Compare your code with the reference there, then do today's task.

Today is a build day, so this lesson runs longer than a weekday's, and it comes in three parts. In the first four chapters you make the call, and learn to read all of what comes back. In the middle three, the code gets a permanent home, a log, and error messages a person can act on. And in the last two, you compare your version with one written for production, and then it's your turn. Keep your editor open, and pause whenever you're typing.

2:11The request

Step one on today's page is the provider's quickstart. Follow it until one call works. That part is theirs to teach, and this lesson starts where it ends. Then make a folder called warm-ups at the root of your repo, and in it a new file: hello_llm.py. Type it from this week's listing. Typing is slower than pasting, and that's the point. Line four reads the model ID from LLM_MODEL, the variable you set yesterday. The second value is a fallback, for a machine where that variable isn't set. Tomorrow's fresh-clone test leans on that fallback. Line five makes the client, and look at what isn't there: a key. The client finds ANTHROPIC_API_KEY in the environment by itself. So the key never has to appear in a file that git tracks.

The rest of the file is one function, called ask, and three calls to it. A prompt goes in and the answer's text comes out. In between is the request. A request needs three things. The model, and then max_tokens: the most output this one request may produce. That's the cap yesterday promised, and here it defaults to four hundred. The third is messages: a list of turns, each with a role and some content. There's one turn here, from the user. A conversation is just a longer list, and tomorrow you send one. And notice the two stars in front of options. Any extra setting you hand to ask is passed straight through to the API. That's how one small function can try a setting nobody wrote a parameter for. Lines eleven to thirteen handle what comes back. Skip them for now, because they get the whole of the next chapter. And at the bottom, one question asked three times: once with the defaults, once with a cap of forty tokens, and once at low effort.

Save the file, and go to the root of your repo. Your key is in .env, where Day one put it. Before you run anything: is that enough? Run it the way the starter's README does, with -m and the module's dotted name. Chapter five explains why. Thirty-odd lines of traceback. Don't read it from the top. In a Python traceback the last line is the verdict, and everything above it is the route the program took to get there. The verdict is a TypeError: could not resolve authentication method. The client was putting the request together, went looking for a key, and found none. Nothing was ever sent.

But the key is right there in .env. It is, and nothing has read it. A .env file is plain text until some piece of code loads it into the environment. If your quickstart call worked a minute ago, the key was most likely exported in that terminal. An exported variable lasts until the terminal closes. The .env file is the copy that's still there tomorrow.

Here's the mistake this error invites. It says it expected an api_key, so you paste your key into the line that makes the client. It runs, and now the key sits in a file that git tracks, one push away from public. Rewind: the key was never missing. It's in .env, and git ignores .env, exactly as Day one set it up. What's missing is the step that reads that file. The fix is two lines. Import load_dotenv, from the python-dotenv package that's already in the starter's dependencies, and call it before anything asks for the key.

Here they are in place. Line three imports it and line five calls it. And the order matters: line five has to run before line six reads the model ID, and before line seven makes the client. One detail from the comment is worth keeping: a variable that's already set wins. If your terminal has its own value for the key, load_dotenv leaves it alone. That comes back in chapter seven. Run it again, and this time the request goes out. So that's a request: a model, a cap, a list of messages, and a key that arrives from the environment and never from the code.

7:12What comes back

You sent a model, a cap and one message. So what comes back? A string would be the natural guess: a question went in, so an answer comes out. It isn't a string. The response has a field called content, and content is a list of blocks. This is the reply to the first of the three calls, as it was recorded. Its list holds one block, the type is text, and the answer is inside. The second thing that comes back is the stop reason: one word for why the model stopped writing. This one says end_turn. The model reached the end of its answer and stopped by itself. And the third is usage, the provider's own token count. Twenty in, ninety-three out. This is the count Day two said to budget from, and from today you record it on every call. Now get the answer out. Try the shortest line that could possibly do it. Take the first block and read its text: resp.content[0].text. It works: there's the answer. And that's what makes this line a trap. It works for this question, on this model, today. So change one of those. Change the question.

Here's the new one: how many days are there from the third of March to the nineteenth of July? Just the number. It's a short question with a short answer. So what comes back this time? Two blocks. The second is the answer: one hundred and thirty-eight. The first has another type: thinking. That's the reasoning you met yesterday, the working a model does before it answers. Nobody asked for it. The model decided this question needed it. Now run that same short line against this reply. The first block, dot text. AttributeError: the thinking block has no attribute called text. That's the real message, from this reply. The line that worked a minute ago now crashes, and your code didn't change. The question did. So ask every block what it is. Keep the ones whose type is text, and join them together. The thinking block is skipped, and the answer comes through. One more thing to read here: usage. Two hundred and eight tokens out, for an answer three characters long. Output counts everything the model wrote: two hundred and five tokens of thinking, and three of answer. That's yesterday's sum in a real response, and all of it is paid from max_tokens.

One more response before the code, and this one is drawn, not recorded. Say the model declines to answer: there's no exception for that. The call succeeds, HTTP two hundred, and the stop reason says refusal. And content can come back as an empty list. Run the short line against that and you get an IndexError: list index out of range. It reads like a bug in your code, when the real story is in the stop reason. The join doesn't crash here. It returns an empty string, and says nothing. That's better than a crash and still not good enough, because an empty string can pass for an answer. So here's the rule, in the guide's own words. Read content blocks by type, never by position, and check the stop reason first.

Now the three lines you skipped make sense. With load_dotenv in the file, they're lines thirteen to fifteen now. Line fourteen prints the stop reason and both token counts, and line fifteen keeps only the text blocks, by type. But hold the rule up against them. The stop reason gets printed, and then the text is returned whatever it said. Keep that in mind until chapter eight.

11:39Three calls

Time to run the three calls, and predict the second one first. The cap is forty tokens, and the question asks for two sentences. You've just seen that a full answer runs to ninety-three. What will the stop reason say?

max_tokens. A stop reason names whatever did the stopping, and here that was your cap, not the model. It didn't finish: it was cut off.

Now run the file. Each call prints its own line first: the stop reason in brackets, then tokens in, then tokens out. On screen is the run from the start of this lesson, so read the same lines in your own terminal. The first call, with the default cap of four hundred, says end_turn. The model finished by itself, with room to spare: ninety-three out of four hundred. The second says max_tokens, and tokens out is exactly forty. That's how a cap looks from outside: the count equals the cap, and the text ends wherever it happened to be. And tokens in is identical on all three lines, because the prompt is identical. Each call pays for it again. Nothing carries over from one request to the next.

Here's that second reply, drawn from the response itself. One text block, cut off. And look at the count: forty out, and every one of the forty is answer. The cap was simply smaller than the answer. Now put the days question under the same cap. Remember what it cost in the last chapter: two hundred and five tokens of thinking before a three-token answer. So what can come back in forty?

No text at all. One block came back, and it's thinking. All forty tokens went on working, and the cap ran out before the answer began. This is a recorded reply too, not a drawing of a risk. And your join returns an empty string. No exception, no warning, HTTP two hundred. In the whole response, the only thing saying that something went wrong is the stop reason. So when the stop reason is max_tokens, treat the text as incomplete, however finished it looks. Then raise the cap, or ask for less.

One call left. The third asks for low effort: yesterday's control, the one reasoning models offer in place of temperature. ask has no effort parameter. The setting rides through those two stars as output_config, and lands in the request untouched. The comment says: where supported. If your model doesn't take it, this line ends in a stack trace. Leave it that way for now, because chapter seven turns that trace into one readable line.

If it does run, put the first printed line beside the third. Same prompt, lower effort. Did tokens out go down? In this run it did: ninety-three, down to seventy-nine. So is that what low effort does? One pair of runs is a hint, not a finding. This file runs three more times today, so keep an eye on those two numbers.

15:29One home for the client

hello_llm.py works. Now think a week ahead: a second script needs to call a model, and then a third. Does each one get its own copy of the client, its own cap, its own way of reading the reply?

The starter answered that before you asked. The llm folder has held one file since Day one, and its first line says what the folder is for: the one way your code calls a model. It even names today. On Day six you move the client out of hello_llm.py, and log every call. So that's the plan: move it first, then log.

Before you move anything, commit what works. And before you commit, run the check that the starter's CI runs on every push: uv run ruff check. One error, in a file that runs perfectly. Ruff wants the imports sorted: Python's own modules first, then installed packages, with a blank line between them. The listing, typed exactly as printed, doesn't pass. The last line tells you the way out: the same command with --fix. One error found, one fixed. It rearranged the imports and left what the code does alone. Now add and commit. If the next half hour goes wrong, this is the version you come back to.

Here's the move. Make a file called client.py in the llm folder, and cut into it everything except the three calls: the imports, load_dotenv, the model ID, the client, and ask. What's left in hello_llm.py is eight lines, and two are new. Line one says what the file does, one step ahead: the logging comes in the next chapter. Line three imports ask from its new home. The three calls ask the same things, with the same settings. They don't know or care where ask lives now.

So far, every run has used -m. Run this one the plain way, the way you'd run any script: Python, then the path to the file. ModuleNotFoundError: no module named llm. But you made that file a minute ago, and the folder is right there. So where is Python looking?

Here's the repo. Your terminal is at the top folder. The script is down in warm-ups, and its third line asks for something called llm. When you run a file by its path, Python starts its search for imports in the folder that file lives in. That's warm-ups. Is there an llm in there? There isn't. The llm folder is next door, one level up, outside the search. Hence the error. Now run it with -m: as a module, by its dotted name. This time the search starts in the folder you're standing in, the top of the repo. From there, llm and warm-ups are both in plain sight. So the rule for this repo is: stay at the root, and run with -m. The starter's README says the same, and so does a comment in pyproject.toml.

Three calls, and the same three kinds of line: end_turn, max_tokens, end_turn. The move changed nothing about how the code behaves. That's what a good move looks like, and it's why you committed first. The counts did move, though. The first line says eighty-four, where it said ninety-three before: same prompt, same settings, and a fresh sample, as Day five said. And the low-effort line says eighty-four as well. So was that gap ever about effort?

19:42A row for every call

Those three printed lines are the only record of what just happened, and the first run's numbers have already scrolled away. Tomorrow you fill a table of experiments, and the site is blunt about it: copy the numbers from your log, not from memory.

So give the client a log, starting at the top of client.py. Lines three to seven import from Python's own library, and four of them are new: one for writing JSON, one for timing, one for timestamps, and one for file paths. Line fourteen is new too. It names the log file once, so the logger further down never repeats the path.

Now ask itself. It gets two new lines. Line thirty-three starts a stopwatch, just before the request. And line forty hands the response, and the time that passed, to a logger. The request between them is the same as before, laid out with one argument to a line. The stopwatch is time.perf_counter. It's the clock Python provides for timing things: it only ever moves forward, even if your laptop's clock gets corrected in the middle of a call. Read it before, read it after, subtract. The difference is the latency in seconds. And what sits between the two readings is the whole request: the trip there, the thinking, the answer, the trip back. That's the wait Day five took apart, measured on your own machine.

Here's the logger. For every call it builds one small dictionary: a timestamp, the model, the stop reason, tokens in, tokens out, and the latency. Look closely at line twenty-one. The model is read from the response, not from your setting. It's the provider's own record of what answered, and that's the one worth keeping. Then it opens a file, calls.jsonl in a folder called logs, in append mode, and writes the dictionary as one line of JSON. Every call adds a line, and nothing already there is touched. The name and the place are yours to choose. One JSON object per line has a name: JSONL. Why not one big JSON list? Because to add to a list you'd have to read the whole file and write it all back. With lines, you only ever append. And if a run dies halfway, every line written before it is still valid.

Run the module once more, then print the file. Three calls, three rows. Read a row the way you'd read a log line at work. When it happened, which model answered, why it stopped, tokens in, tokens out, and how long it took. Now find the cut-off call, and compare its latency with the other two. It wrote half as many tokens, so you'd expect it to be the fastest. In this run it's the slowest: one point two seconds, against about one. A single row tells you how long one call took, not how long calls take. For that you need many rows, and now you're collecting them. And those two numbers again. First row, seventy-nine out. Third row, the low-effort call, eighty-four. In chapter four, low effort looked fourteen tokens cheaper. Here it costs five more.

One mistake is easy to make here: putting the logging in the script. hello_llm.py calls ask, then writes the row itself. It works today, and next week a second script calls ask and nobody remembers the row. Rewind to where the log line sits: inside ask, in the client. A caller can forget to log. The client can't. That's the real case for one home. Whatever every call needs goes there once: the log today, and in week two a timeout, retries, and the cost of each call.

24:20Errors you can read

Day five promised you an error caused on purpose. Start with the easiest one to cause: a wrong key. Put a made-up value for ANTHROPIC_API_KEY in front of the command, for this one run only. This time the request reaches the provider, and the provider says no. Remember the detail from load_dotenv: a variable that's already set wins. So the made-up key on the command line beat the real one in .env. Twenty-odd lines again, and only the last one matters: AuthenticationError, code four oh one. Now picture this in a teammate's terminal, or a production log. They needed one line and got a wall.

Three errors are worth catching by name today, and each has a number. Four oh one: the key was rejected. You've just met it. Four oh four means not found. Here that's the model ID: whatever LLM_MODEL holds isn't a model this provider has. And four hundred is a bad request. It reached the API, and the API won't run it as written. A setting your model doesn't support lands here, which is yesterday's temperature story. What do the three have in common?

Send the same request again and you get the same error, every time. None of them is worth a retry: each one waits for a person to change something. The SDK already behaves that way. It quietly retries rate limits and server errors a couple of times, and it never retries these three.

So here's the last change to hello_llm.py. The calls move inside a try block, and a fourth call joins them. The fourth one sets a temperature. It goes through extra_body, because current versions of the SDK no longer take temperature as a keyword. On the latest models, this is the call that comes back four hundred. Under the calls sit three except lines, one for each error. A bad request prints the message the API itself sent back. Model not found prints the ID it tried, so a typo is right there on screen. And a rejected key names the variable to check. Notice what isn't here: a catch-all. Anything you didn't plan for should still crash loudly, with its full trace. An except that swallows everything and prints 'something went wrong' hides exactly the bugs you most need to see. And why catch here, in the script, and not inside ask? Because what to do about an error depends on who's calling. A warm-up prints a line and stops; the client's job is only to raise.

Same wrong key, same command, one line. It says what happened, and it says where to look.

The four oh four is yours to cause. Put a model name that doesn't exist in LLM_MODEL, in front of the command, and read your own line.

The four hundred needs no trick at all. Run the file clean, with your real key. Three answers, as before, and then one more line. Bad request, four hundred: temperature, extra inputs are not permitted. That's the API's own sentence, printed by your except line. This model doesn't take a temperature. It's the error Day five promised, and it arrived as one line. And if your model does accept a temperature? Then a fourth answer prints here, in place of that line. Write that down too. And those two counts, one last time: seventy-eight, and seventy-nine. Across four runs, low effort came out fourteen tokens shorter, then level, then five longer, then one longer. Stop after the first run, and you'd have reported a saving that the next three don't show. Tomorrow you measure it properly: one setting at a time, and no claim without a number.

One thing to expect on your own machine: a try block ends at the first failure. If your model rejects the effort setting, the run stops at call three, and the temperature call never happens. Take effort out, and run again.

29:11Compare, then fix

Your version works. Now put it beside one that was written for production. The last step on today's page points to the companion repo: a folder called guide-04-production, and in it a file, text.py. The first thing this function does is the thing yours doesn't. It checks the stop reason. On refusal or max_tokens it raises an exception called NoText, and never looks at the content. Then comes the join. It's your line, character for character: text blocks only, picked by type. And one last guard. If what's left is empty, or only whitespace, that's NoText as well. An empty string never leaves this function dressed as an answer. The docstring gives the reason in one sentence, and by now you've watched both halves of it happen. The first block can be a thinking block. And a refusal can come back with no text at all.

Beside it is a short script, block_types.py. Its docstring says how to run it, and that it makes one real call. Line fourteen is the reason it exists. It prints the type of every block in the response, in order. Run it against your model: does the list start with text, or with thinking? Whichever you see, don't build on it. You've watched one model answer one question with a single block, and another with two. The order of the blocks belongs to the model, and it isn't yours to count on. The type is.

So what does your version miss? Go back to the forty-token call. ask printed max_tokens, and then returned half a sentence as if it were the answer. A person watching the terminal notices. A program calling ask never will: it gets a string, and one string looks much like another. Fixing that is today's last step, and there's a real decision inside it. The reference raises on max_tokens. But your warm-up wants to see the cut-off text, because that's the experiment.

So what should ask do? It could raise, like the reference. Then no caller can mistake half an answer for a whole one, and your warm-up has to catch the exception to see the cut-off text. It could return the stop reason along with the text. Every caller gets the whole story, and every caller has to remember to read it. Or it could take a switch: strict by default, and partial text only when a caller asks for it. That's one more parameter to get right.

There isn't one right answer. Write what you chose and why in LEARNING.md, and make sure an empty answer never passes for a real one.

32:24Your turn

Your turn, and today it's a real build, so give it the time it needs. Start with the quickstart, until one call works. Then type hello_llm.py from the listing, add load_dotenv, and run the three calls. Read every printed line before you move on. Move the client into the llm folder, run with -m from the root, and add the log: one line of JSON for each call, with its latency. Catch the three errors by name. Then cause each one on purpose, so you've watched your own line print. Open the reference, compare, and fix what yours misses. The decision goes in LEARNING.md. On the student track, do it once more against a second provider, or a local model with Ollama. The field names change. The questions you ask of a response don't.

Before you commit, run the usual checks. Ruff first, and it passes. Then the starter's own tests. Eleven passed, so nothing you added today broke what was already there. And git status. The warm-up changed, client.py is new, and so is a folder called logs. Stop on that last line. A log is output, not source, and it grows with every run.

Adding the logs folder to .gitignore is a sound default, since the numbers you keep get committed tomorrow, in EXPERIMENTS.md.

On the map, two folders moved today. The warm-ups folder has its first file, and the llm folder has a client that every later week calls through.

Done when, exactly as the site puts it. uv run python -m warmups.hello_llm prints answers with stop reasons and token counts, each call adds a JSONL row with model, stop reason, tokens and latency, and an unsupported parameter prints one clear line instead of a stack trace.

Self-check one. Why is reading the first block's text an unsafe way to get the answer? Because the first block is your own prompt, echoed back; because the first block can be thinking, or missing; or because the text field is empty until you log it?

The first block can be thinking, or missing altogether. Blocks come back in the model's order, so check the stop reason, then join the blocks whose type is text.

Self-check two. The stop reason is max_tokens, and the text is empty. What happened: thinking used the whole output cap; the model refused the request; or the key ran out of credit?

Thinking used the whole cap. Thinking counts toward max_tokens, so the cap ran out before the answer began. Raise it, or ask for less.

Self-check three. A call fails with a four hundred. What should your code do: retry it three times, with a delay; fall back to a different model; or print one clear line, then stop?

Print the line and stop. A four hundred means the request itself is wrong, so a retry fails the same way every time, and a fallback would only hide the bug.

That's Day six: a client, a log, and errors you can read. Tomorrow is Day seven, and you use all three: four experiments with one setting changed at a time, a fresh-clone test, and your first tag, v0.1. This lesson was narrated with an AI voice.

How to do it

  1. 1Make one call by following the quickstart, then write warmups/hello_llm.py yourself from this week's listing: model ID from LLM_MODEL, key from the environment, and only the text blocks kept.
  2. 2Run its three calls and read each printed line: the default, max_tokens=40 (expect stop_reason max_tokens and a cut-off or empty answer), and output_config={"effort": "low"} where your model supports it.
  3. 3Move the client into llm/ (for example llm/client.py) and log every call as one JSON line: timestamp, model, stop reason, input and output tokens, and latency measured with time.perf_counter(). Run scripts as modules from the repo root, uv run python -m warmups.hello_llm, so import llm resolves.
  4. 4Make errors readable: catch anthropic.BadRequestError for an unsupported parameter (try extra_body={"temperature": 0.2}), anthropic.NotFoundError for a bad model ID and anthropic.AuthenticationError for a bad key, and print one clear line for each.
  5. 5Then compare with guide-04-production: app/llm/text.py checks the stop reason first and reads blocks by type, and scripts/block_types.py shows why. Fix anything your version misses.
Done when

uv run python -m warmups.hello_llm prints answers with stop reasons and token counts, each call adds a JSONL row with model, stop reason, tokens and latency, and an unsupported parameter prints one clear line instead of a stack trace.

Common mistake

Reading resp.content[0].text. The first block can be a thinking block and a refusal may carry no text, so check stop_reason first and join only the blocks whose type is text.

This week's build

your first logged calls

Move the client and logger into llm/. Vary max_tokens, the system prompt, and effort and temperature where supported. Responses can hold several blocks (thinking, text), so keep only the text; at 40 tokens it may be cut off or empty. Try temperature only where supported: current Python SDKs no longer take it as a keyword, so pass extra_body={"temperature": 0.2}, and the latest Claude models return a 400 error for it, so catch anthropic.BadRequestError. On OpenAI, use responses.create(...) and resp.output_text.

warmups/hello_llm.pypython
import os
import anthropic

MODEL = os.getenv("LLM_MODEL", "claude-haiku-5-5")  # set to a current model ID
client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment

def ask(prompt: str, max_tokens: int = 400, **options) -> str:
    resp = client.messages.create(
        model=MODEL, max_tokens=max_tokens,
        messages=[{"role": "user", "content": prompt}], **options)
    u = resp.usage
    print(f"[{resp.stop_reason}] in={u.input_tokens} out={u.output_tokens}")
    return "".join(b.text for b in resp.content if b.type == "text")

q = "In two sentences, what is a context window?"
print(ask(q))
print(ask(q, max_tokens=40))                    # watch stop_reason
print(ask(q, output_config={"effort": "low"}))  # effort, where supported

Create a free account to mark steps done and keep your progress. Week 1 is free.

Start free