PixioLab
Day 11–1.5 h

Setup

Read Chapters 01–02; install uv; create the repo; set a spend limit.

Student track adds

Intro to LLMs (1 h).

Read the transcript

0:00One repo, 45 days

This is a repo called sourcebook, on Day forty-five. It answers questions about real documents. It cites its sources, and it proves it works, with numbers. It was not built in one go. It grew one layer a week, and a tag marked each layer, from v0.1 to v1.0. Today it is empty. By the end of this lesson it will be live on GitHub, installed, tested, and safe to make public. So here is the question for today. What does a repo need on day one, so that forty-four days from now you can hand the link to an interviewer without a second thought?

Three outcomes. First, you can say what you are building over the next forty-five days, and why this combination gets an interviewer's attention. Second, you have a public repo that installs from a fresh clone and passes its tests and its lint. Third, you have one API key for this project, kept out of git, with a spend limit behind it.

A fast lane, before we start. If you have read Guide zero and know the capstone and the nine rules, jump to chapter five of this video, where the hands-on work starts. If Python three point eleven and uv are already on your machine, the first half of Chapter five is a check, not a setup. And if you have called an LLM API from code and can explain tokens and sampling, Guide zero says do all of Week one in two sittings: set up the repo, run the build, move on.

You will hear three names all the way through this path. They are three parts of one system, and each has one job. Sourcebook is the system. It is retrieval-augmented generation, RAG: a model answers questions from documents you give it, and cites the passages it used. Hybrid search means two indexes, a vector index and a keyword index, searched together; permission-aware means it only retrieves what the asking user is allowed to see. Groundtruth is the evidence. A set of questions with known right answers, called a golden dataset, and a harness that reports recall@k, MRR and hit rate at k for every retrieval configuration you try. All three score how well retrieval finds the right passage, and Week five teaches each one. Tripwire is the discipline. A gate in continuous integration that blocks a pull request when quality or safety gets worse. How much of it you build depends on your track, which you pick in chapter three. Why this combination? RAG over internal documents is a pattern you will meet in many AI engineering roles, and measurement is the part most portfolio demos leave out. An interviewer gets three things to probe: retrieval design, evaluation method and engineering discipline.

3:18The map

This is the picture that Weeks three to six fill in, one part each. Three lanes. Before we walk it, hold one question: where in this picture should permissions be enforced? The top lane runs offline, before any user shows up. You take a corpus of documents, with its license noted, parse them, and chunk them into passages. Each passage is embedded with a local model, so this step costs nothing in API calls. The index stores three things per passage: its vector, its keyword terms for BM25, and metadata on who may read it. The middle lane is live. A user arrives with their group memberships, and input checks cap the size and rate of requests before anything expensive runs. Here is the answer. The retriever filters by permission first, then runs hybrid search and reranks. The model never sees a passage the user may not read, because the retriever never fetched it. The model writes an answer with citations, and output checks confirm every citation points at a passage that was really retrieved. Every request also writes a trace: chunk ids, tokens, latency and cost. The bottom lane is evaluation. Groundtruth runs golden questions and attack prompts through the same pipeline users hit, and Tripwire turns the numbers into a gate in CI. If a pull request makes them worse, it is blocked.

Now the same picture as folders. This is the layout from Guide zero, Chapter two, and your repo grows into it. The template starts you with three of these folders, llm, sourcebook and groundtruth, each still empty. Weeks one and two build shared plumbing: an LLM client with retries, timeouts and a cost log, two warm-up scripts, and prompt files kept under version control. Week three fills sourcebook: ingest, chunk, index, retrieve, answer. Week four adds an agent folder inside it. The corpus you choose in Week two is described in data slash README. Week five is Groundtruth: the metrics, the runner, the judge, the golden questions in eval, and one JSON file per run in results, committed. Week six adds the Tripwire workflow under .github, plus a small classic machine-learning notebook beside the core. The README is written last, on Days forty-three to forty-five, results first. One file starts today: LEARNING.md, your decisions log. We come back to it in the last chapter.

Each week ends with a tag. Weeks one and two, v0.1 and v0.2, build the LLM client, the cost log, prompt files and a retry wrapper, and you choose your corpus. Week three is RAG: ingest, index, hybrid search, cited answers, tagged v0.3. Week four adds agent mode with budgets and traces, v0.4. Week five is evaluation and operations: Groundtruth, fifty or more golden questions, cost and p95 latency. That is v0.5, and it is the week you never compress. Week six adds the permission tests, guardrails, an attack set and the Tripwire gate in CI. That is v0.6, and by Day forty-two the system is feature-complete. The last three days go on polish and publishing, not panic: a results-first README, a demo, a deploy, tagged v1.0. Keep a short CHANGELOG.md as you go, so a reviewer can see which change moved which number.

7:59Time and money

Before any command, one decision: which track. Both reach the same repo on Day forty-five. They differ in hours a week, and in how much of Tripwire you build. The working-engineer track assumes a full-time job, so evenings and weekends. The student track is for full-time students, career breaks, anyone with open days. Working engineers plan ten to twelve hours a week: about an hour to an hour and a half each weekday, and two and a half to three hours on each weekend day. Students plan twenty to thirty hours: three to five hours a day, six days a week, one day off. Over the forty-five days that is about sixty-five to seventy-five hours on the working track, and one hundred thirty to two hundred hours on the student track. By Day forty-five the working track publishes Sourcebook and Groundtruth with a minimal Tripwire gate. The student track builds the full Tripwire and adds two or three stretch items.

The day tables assume Day one is a Monday, so Days six and seven are the long weekend blocks, and the same pattern repeats every week. If the working track takes you eight or nine weeks, that is normal. Calendar time can stretch; a week with no build is a week you can't show anyone.

You will fall behind at some point; most people do. Guide zero gives the order to drop things. Optional resources go first. Then the student extras, if you took them on. Never the week's build, and never the evaluation. The build is what you can show, and the evaluation is what makes it believable. And Day forty-two is a buffer, built into the plan. Use it.

Prerequisites first. You need Python, including classes, type hints, virtual environments and reading a stack trace. You also need Git and GitHub, HTTP and JSON, and the command line. SQL, Docker and basic statistics help. No machine-learning background is needed, and if Python is the gap, spend a week on it first.

Then, what you can compress. If you can explain tokens and sampling, and you have called an LLM API from code, do Week one in two sittings. If you can get schema-valid JSON from a model and run a tool-call round trip, do the Week two build in a weekend and give the spare days to Week three. If you have built RAG and measured recall@k on labeled questions, build Sourcebook fast and start labeling Groundtruth's questions early. If you ship agents with tool loops in production, do only Week four's MCP and comparison work, and give the time to Week five. And whatever you can already do: never compress Week five. Evaluation turns a demo into evidence.

Now money. Most of what you use on this path is free: the courses, videos and docs it links to, open-source tools such as Chroma, pgvector, Ragas and promptfoo, public GitHub repos with Actions, and Ollama for local models. Some need a free account. Some tools are freemium: LangSmith, Langfuse Cloud, Hugging Face Spaces and the Gemini API free tier. One caveat: Gemini's free tier may use your content to improve Google's products, so send it no private data. What costs money is API use: model calls, priced per million input tokens and per million output tokens. Pay as you go, and no GPU spend is needed anywhere in this plan.

Predict first. Fifty questions, each about three thousand input tokens and three hundred output tokens, at an example price of two dollars per million input and ten dollars per million output. Is one run about five cents, forty-five cents, or four dollars fifty?

Forty-five cents. One hundred fifty thousand input tokens cost thirty cents, and fifteen thousand output tokens cost fifteen cents. Thirty runs come to about fourteen dollars, which is why repeated evaluation is the biggest cost in the plan, and why you make it cheap.

Plan on roughly ten to forty dollars of API usage across the forty-five days, if you develop on a small model and use a larger one only for final eval runs and judging. Heavy experimenting with frontier models can push that toward fifty to one hundred dollars.

Four habits keep it low. Develop on a small, fast model, and run the final numbers on the one you would ship. Embed locally with Sentence Transformers. Retrieval-only metrics such as recall@k then cost nothing, because no hosted model is called. Run non-urgent evals through a batch API. Anthropic's is billed at half price. Cache long system prompts, and cap the maximum output tokens on every call, so one runaway answer cannot run up the bill.

If you have no budget at all, there is a path. Ollama runs small open-weight models on your own machine. They are slower and weaker than hosted frontier models, and tool calling varies by model, but every build in this path still works. All retrieval work, including Groundtruth's retrieval metrics, needs no paid API at all. One warning that applies to everyone. Never put a key without a spend limit behind a public demo, and add a per-session query cap. You set that limit later today.

14:40Nine rules

Guide zero sets nine rules for the whole path. They group into four: shipping, evidence, craft and safety. Two of them get a closer look in a moment. Rule one: ship every week. A rough version that runs beats a perfect one in your head, so tag a release each Sunday. Rule three: one repo, one story. Every build feeds the capstone, and one finished repo beats three half-done ones. Rule five: don't skip the evaluation week. When time runs short, compress any other week first. Rule two: no claim without a number. Not 'it works', but 'recall@10 of 0.84 on fifty labeled questions'. We come back to this one. Rule four: write decisions down. Log the options, your choice and the evidence in LEARNING.md. Rule nine: check fast-moving facts at the source. Models, prices and SDKs change every few months, so trust the docs over tutorials, and over this video too. Rule six: use the engineering you have. Tests, types, logging and small commits are your edge over people who only prompt. Rule seven: use AI coding tools, and own every line. We unpack this one next. Rule eight: protect keys and data. Keys live in environment variables, and no private data goes to a third-party API without permission. Chapter six is this rule, hands-on.

Here is rule two in practice. 'It works.' Say that to an interviewer and you have told them nothing: not what you measured, not on what, not how well. Rewind, and ask what the sentence is missing. Which metric? Measured on which data, and on how many items? Now the same claim with its number: 'recall@10 of 0.84 on fifty labeled questions'. Recall at ten asks how often a correct passage landed in the top ten results, so this says eighty-four percent of the time, on fifty questions you labeled yourself. That number can be checked, compared and improved. Every number in your README and your log carries its n in this way: the metric, the value, and how many items it was measured on.

Rule seven is not a ban on assistants; it is a division of labour. Let an assistant write boilerplate, tests and refactors. You make the design decisions, and you write the eval cases. Those are the parts an interviewer will probe. And you read every diff. If you can't explain a line, you can't ship it, and an interviewer will ask. The starter template repeats this rule in its README, because it applies from Day one.

18:01Your machine, your repo

Hands on, from here to the end. Open a terminal and check Python. You need three point eleven or newer; this machine has three point twelve, which is fine. Then uv, the Python package and project manager this path uses for environments, dependencies and running code. If it is missing, the install is one line from its docs. That line is marked example here, and its output is left out, because the installer prints differently on each machine. Check that uv answers. Any recent version is fine; the one here is what was installed when this was captured, in November twenty twenty-six.

Pause the video here, and get both to answer on your machine. The rest of this lesson clones a repo and installs it with uv, so nothing works without these two.

Now the repo. Open the sourcebook-starter template on GitHub, under pixiolab-ai: it holds the first folders of the layout from chapter two, a test workflow and a precheck script, with none of the work done. This frame is an illustration; your screen will look a little different. Click 'Use this template', then 'Create a new repository'. Name it sourcebook, and make it public. Public matters: the checks on pixiolab.com read public repos only, and a public repo is the thing you hand to an interviewer. Create it. You now have your own sourcebook repo under your own name, generated from the template, and every day from here on adds to it.

Clone it, with your own GitHub name in the address, and step into the folder. The clone line is marked example, because its output depends on your account. Run uv sync. It reads pyproject.toml, creates a virtual environment in .venv, resolves the dependencies and installs them: twenty-three packages here. It also writes uv.lock, the exact versions it resolved, and you commit that file today. Now the first check: uv run pytest, with dash q for quiet, runs the starter's tests inside that environment. Eleven passed, with no API key and no network. This output is real, captured in November twenty twenty-six. And the second: uv run ruff check, with a dot for the current folder, is the linter. All checks passed. A fresh clone that installs and passes both checks is outcome two, done.

A look at what uv just read. Line six of pyproject.toml requires Python three point eleven or newer, which is why we checked your version first. Three runtime dependencies: the anthropic SDK, pydantic, and python-dotenv, which loads .env into the environment. The template has already done uv init and uv add for you; if you use another provider, swap in its SDK with uv add. And a dev group: pytest and ruff, the two commands you just ran. uv sync installs them too, and CI runs the same two on every push.

Pause here and do the same: create the repo, clone it, sync, and run both checks. Eleven passed and all checks passed is what success looks like. If either fails on a fresh clone, fix it now, because everything later builds on this.

22:02Lock down your key

The question for this chapter: how does a key end up in a public repo, and how do you make that impossible before it can happen? Start with the file that answers it, .gitignore. Its first four lines are about secrets. Line two ignores .env, the file that will hold your real key. Line three ignores every file whose name starts with .env and a dot, so a variant can't slip in either. Line four, with the exclamation mark, un-ignores .env.example. That is the one file you do commit: the same variable names, with empty values, so anyone cloning the repo knows what to set. Further down, .venv is ignored too, so the environment uv built stays off GitHub. These rules exist before your first commit, and that ordering is the whole point of starting from the template.

Here is .env.example, all five lines. The comment on line one is the command you are about to run, and line two is the rule: never a real key in this file. Line three is the key, empty. In your .env you will paste a key you created just for this project, so that if it ever leaks you revoke one key and nothing else breaks. Line five names the model to develop on, a small, fast one, as the budget advice said. Your code will read the model from here, never hard-coded.

First make a key in your provider's console, new, just for this project. Then copy the example to .env, open .env in your editor, and paste the key on line three, after the equals sign. Then prove git will ignore it. git check-ignore, with dash v, answers with the rule that matched: .gitignore, line two, .env. If this prints nothing, stop: the file is not ignored. And the check you will repeat all path long: git status. The only untracked file is uv.lock, which you commit in the next chapter. .env is not listed, and it must never be.

Here is the mistake the Day one page warns about. Someone starts from an empty folder, writes .env, and runs git add, commit and push before any ignore rule exists. The key is now in a public repo, and it is in the history, so deleting the file later does not remove it. Rewind. If this ever happens to you, the fix is not a new commit; it is revoking the key in your provider's console, immediately. Then repair the ignore rule. Played forward the right way: the ignore rule exists before the first commit, which the template guarantees, and git status never lists .env. One key per project, so a leak costs you one key. And a spend limit behind it, so a leak costs a bounded amount of money. That limit is next.

Open your provider's console and find billing, or spend limits. This frame is an illustration; every console looks different, and some offer an alert rather than a hard limit. Set whichever yours has. Pick a monthly number that fits your plan. The guide budgets ten to forty dollars over the whole forty-five days, so a limit around twenty dollars a month leaves room to work without leaving you exposed. Save it. Outcome three is done: one key, out of git, with a limit behind it. Today's Done when asks for exactly this: your console shows a spend limit or an alert.

Pause and do these four. Create the key, copy the example and paste the key in, check the ignore rule and git status, and set the limit. Then come back for the last chapter, which is short.

26:33Your log, and proof

Open LEARNING.md. The template gives you the headings; you give it three lines a week, written before you tag. First, what you built: what runs now that didn't last week. Second, one decision with its evidence: the options, your choice, and the number or observation behind it. Third, one thing that broke: what failed, how you found it, and the fix, or why it is still open. Notes the days ask for, such as the Day two token table, go under that week's three lines. Week one's heading is already here, under v0.1. This log feeds two things: your Day forty-three README, and the stories you tell in interviews. Three honest lines a week beats reconstructing six weeks from memory later.

Last commands of the day. Stage uv.lock, the lockfile uv sync wrote. Committing it means a fresh clone installs exactly these versions, which is what the Week one checkpoint tests for. Commit it with a short message that says what changed. One file, five hundred fifty-two lines of pinned versions. And push. The push line is marked example, because it shows your account. Your repo on GitHub now has pyproject.toml, .gitignore, .env.example, LEARNING.md and uv.lock, and no .env.

Back to the map. Today lit one file, LEARNING.md. Everything else in this tree is dark on purpose; each part has a day that fills it. This week fills llm and warmups, and ends with the tag v0.1 on Day seven.

Done when, exactly as the site puts it. Your public repo shows pyproject.toml, a .gitignore that lists .env, .env.example, and LEARNING.md. git status never lists .env. And your provider's console shows a spend limit or an alert. If all three are true, Day one is done.

Self-check one. A key was committed and pushed to your public repo an hour ago. What do you do first: revoke the key at your provider, delete .env and push a new commit, or make the repo private?

Revoke the key, now. Deleting the file leaves it in the history, and hiding the repo does not undo what was public, so both other options leave a live key. Revoke first, then fix the ignore rule.

Self-check two. You are behind, and Week five is coming. Which do you compress: Week five itself, the weekly tag, or any other week but never Week five?

Any other week first, never Week five. Rule five and the compress table say the same thing: evaluation turns a demo into evidence. And the tag stays, because a week with no shipped build is a week you can't show anyone.

Self-check three. Which of these claims gives everything rule two asks for? 'Hybrid search is thirty percent better than vector-only.' 'recall@10 of 0.84 on fifty labeled questions.' Or, 'the answers are eighty-four percent accurate now.'

The second: it names the metric, the value and the n. Thirty percent better names no metric and no data, and eighty-four percent accurate does not say on which questions, or on how many. A number alone is not enough.

That is Day one. Tomorrow is Day two, Tokens: you paste English, code and a second language into a tokenizer, and see why the same content costs different amounts, and why a model struggles to count the letters in strawberry. This lesson was narrated with an AI voice.

How to do it

  1. 1Read Guide 00's first two chapters. Pick your track, note which weeks you can compress, and study the repo layout you'll grow, so you know where each week's work lives: llm/, prompts/, warmups/, sourcebook/, eval/.
  2. 2Install Python 3.11 or later and uv. Open the sourcebook-starter template on GitHub, choose Use this template, and create a public repo named sourcebook. Clone it and run uv sync. To start empty instead, run uv init sourcebook, cd sourcebook and uv add anthropic pydantic, then create a public repo for it. Swap in your provider's SDK if you use another.
  3. 3Before your first commit, check that .gitignore lists .env (the template's does). Keep a .env.example with placeholders such as ANTHROPIC_API_KEY= and LLM_MODEL=, and never put a real key in it.
  4. 4Create one API key just for this project and put it in .env or your shell, never in code. Set a monthly spend limit or budget alert in your provider's console.
  5. 5Open LEARNING.md (the template has one; otherwise create it) and keep three lines under each week's heading: what you built, one decision and why, one thing that broke. Commit and push.
Done when

Your public repo shows pyproject.toml, a .gitignore that lists .env, .env.example and LEARNING.md; git status never lists .env; and your provider's console shows a spend limit or alert.

Common mistake

Committing the key before .env is ignored. Add the ignore rule first, and if a key ever reaches GitHub, revoke it in the console: deleting the file leaves it in the history.

Create a free account to mark steps done and keep your progress. Week 1 is free.

Start free