Before Day 1: plan, rules and costs
Before Day 1, pick a track, check the prerequisites, decide what you can compress, set up your machine, and know what this will cost. Most of it is free, but calling a model costs a little.
Two tracks, one destination
Day tables assume Day 1 is a Monday, so Days 6–7, 13–14 and so on are the long weekend blocks (marked Weekend). If the working track takes eight or nine weeks, that's normal: calendar time can stretch, but a week with no build is a week you can't show anyone.
| Working-engineer track | Student track | |
|---|---|---|
| Who it's for | A full-time job, so evenings and weekends | Full-time students, career breaks, anyone with open days |
| Weekly time | 10–12 h: about 1–1.5 h each weekday, plus 2.5–3 h on each weekend day | 20–30 h: 3–5 h a day, six days a week, one day off |
| Over 45 days | About 65–75 h | About 130–200 h |
| Done by Day 45 | Sourcebook + Groundtruth published, with a minimal Tripwire gate | Sourcebook + Groundtruth + full Tripwire, plus two or three stretch items |
| If you fall behind | Drop optional resources first, then the student extras. Never drop the week's build or the evaluation. Day 42 is a buffer. | Drop optional resources first, then the student extras. Never drop the week's build or the evaluation. Day 42 is a buffer. |
Prerequisites, and what you can compress
You need: Python (classes, type hints, virtual environments, stack traces); Git and GitHub (branches, pull requests Pro Git); HTTP and JSON; and the command line. SQL, Docker and basic statistics help. If Python is the gap, spend a week on it first. No ML background is needed.
| If you can already… | …then |
|---|---|
| Explain tokens and sampling, and you've called an LLM API from code | Do Week 1 in two sittings: set up the repo, run the build, move on. |
| Get schema-valid JSON from a model and run a tool-call round trip | Do the Week 2 build in a weekend; give the spare days to Week 3. |
| Build RAG and you've measured recall@k on labeled questions | Build Sourcebook fast, then start labeling Groundtruth's questions early. |
| Ship agents with tool loops in production | Do only Week 4's MCP and comparison work; give the time to Week 5. |
| Anything at all | Never compress Week 5. Evaluation turns a demo into evidence. |
Day 1 setup
- Python and uv. Python 3.11+. Create your repo from the sourcebook-starter template, then run
uv sync. Or start empty:uv init sourcebook, thenuv add anthropic pydantic. - A public repo, today. Git-ignore
.envbefore the first commit (the template already does). - One API key for this project. In
.envor your shell, never in code. Commit a.env.examplewith placeholder values. - A spend limit. Set a monthly limit or budget alert in your provider's console, if it offers one.
- A learning log. Each week in
LEARNING.md: what you built, one decision and why, one thing that broke. It feeds your README and interview stories.
Nine rules of the road
- Ship every week. A rough version that runs beats a perfect one in your head. Tag a release each Sunday.
- No claim without a number. Not "it works" but "recall@10 of 0.84 on 50 labeled questions".
- One repo, one story. Every build feeds the capstone; one finished repo beats three half-done ones.
- Write decisions down. Log the options, your choice and the evidence in
LEARNING.md. - Don't skip the evaluation week. When time runs short, compress any other week first.
- Use the engineering you have. Tests, types, logging and small commits are your edge over people who only prompt.
- Use AI coding tools, and own every line. Let an assistant write boilerplate, tests and refactors. You make the design decisions, write the eval cases and read every diff. If you can't explain a line, you can't ship it, and an interviewer will ask.
- Protect keys and data. Keys in env vars; no private data to third-party APIs without permission.
- Check fast-moving facts at the source. Models, prices and SDKs change every few months; trust docs over tutorials.
What's free, what's freemium, what costs money
| Cost | What falls in it | Caveat |
|---|---|---|
| Free | Courses, videos and docs here (except below); open-source tools such as Chroma, pgvector, Ragas, promptfoo, garak, Presidio and self-hosted Langfuse; public GitHub repos and Actions; Streamlit Community Cloud; Ollama. | Some need a free account (Kaggle, Hugging Face). Streamlit Community Cloud hosts public apps only. |
| Freemium | LangSmith (free Developer plan), Langfuse Cloud (free Hobby plan), Hugging Face Spaces, the Gemini API free tier. | Gradio and Docker Spaces need a paid plan. Gemini's free tier may use your content to improve Google's products: send no private data. |
| API use | Model calls, priced per million input and output tokens, including Anthropic's tutorial and course notebooks. | Pay as you go; see the budget below. No GPU spend is needed in this plan. |
A realistic API budget
Plan on roughly $10–$40 of API usage over the 45 days if you develop on a small, fast model and use a larger one only for final eval runs and judging. Heavy experimenting with frontier models can push that toward $50–$100. Repeated evaluation runs are the biggest cost, so make them cheap.
Illustrative cost of one eval run. Take 50 questions at about 3,000 input and 300 output tokens each: 150k input and 15k output tokens. At an example price of $2 per million input and $10 per million output tokens, that's $0.30 + $0.15 = $0.45 a run, or about $14 for thirty runs. Retrieval-only metrics such as recall@k, computed with local embeddings, cost nothing. Copy real prices from your provider's pricing page, for example Claude pricing.
The zero-budget path. Run small open-weight models locally with Ollama. They are slower and weaker than hosted frontier models, and tool calling varies by model, but every build still works. All retrieval work, including Groundtruth's retrieval metrics, needs no paid API.
Watch out. Never put a key without a spend limit behind a public demo, and add a per-session query cap.
Keep it low:
- Develop on a small model; run final numbers on the one you'd ship.
- Embed locally with Sentence Transformers.
- Run non-urgent evals through a batch API; Anthropic's is billed at half price.
- Cache long system prompts; cap
max_tokens.
The capstone
You build one system across the 45 days, adding a layer each week. By Day 42 it is feature-complete, so the last three days go on polish and publishing rather than panic.
Starter and companion code. The starter template gives you this chapter's repo layout, a test workflow and a precheck script, with none of the work done: github.com/pixiolab-ai/sourcebook-starter. Reference implementations for Guides 01–04 and 07 are in github.com/pixiolab-ai/pixiolab-ai-mentor. Use them to check your build, not to replace it: your capstone should be your own code.
- Sourcebook is the system: hybrid-search RAG with citations and permission-aware retrieval over a document corpus.
- Groundtruth is the evidence: a labeled golden dataset and a harness that reports recall@k, MRR and hit rate@k for each retrieval configuration.
- Tripwire is the discipline: a CI gate that blocks a pull request when quality or safety regresses. It is a full build on the student track and a minimal gate on the working track.
- Tag a release at the end of every week (
v0.1tov0.6, thenv1.0) and keep a shortCHANGELOG.md. A reviewer can then see how the system grew and which change moved which number.
Create a free account to mark steps done and keep your progress. Week 1 is free.
Start free