PixioLab

Glossary

AI engineering, in 40 terms

The words you meet building with language models, each defined in a few sentences and grouped by the week of the 45-day path that teaches it.

By PixioLab

Week 1: How LLMs work, and your first call

Token

A token is the unit a language model reads and writes: a word, part of a word, a punctuation mark or a few bytes. Models are priced, limited and timed in tokens, not words. English averages about four characters per token; code and many other languages need more tokens for the same content.

Taught on Day 2: Tokens, free to watch

Tokenizer

A tokenizer splits text into tokens from a fixed vocabulary, usually learned with byte-pair encoding (BPE), so common words become one token and rare strings several. Each model family has its own tokenizer, so the same text can cost a different number of tokens on different models.

Taught on Day 2: Tokens, free to watch

Context window

The context window is the most tokens a model can handle in one request: the system prompt, the conversation so far, any documents and the reply it writes. A chat re-sends its history on every turn, so long conversations cost more per turn and eventually hit the limit.

Taught on Day 2: Tokens, free to watch

Pretraining

Pretraining teaches a model to predict the next token across a very large body of text. It produces a base model that knows a lot about language and the world but doesn't follow instructions well; post-training then turns it into an assistant.

Taught on Day 3: How models are built, free to watch

Post-training

Post-training turns a base model into an assistant: supervised fine-tuning on example conversations, then reinforcement learning from human or AI feedback toward helpful, safe answers. It shapes behavior and style far more than it adds knowledge.

Taught on Day 3: How models are built, free to watch

Hallucination

A hallucination is a fluent, confident answer that is false or unsupported, such as an invented citation or API. It happens because a model generates likely text, not checked facts. Grounding answers in retrieved sources, requiring citations and letting the model say it doesn't know all reduce it.

Taught on Day 3: How models are built, free to watch

Transformer

The transformer is the neural network design behind modern language models. In each layer, every token gathers information from the tokens before it (attention) and is then transformed by a feed-forward step. Dozens of stacked layers build up meaning, and the last one scores every possible next token.

Taught on Day 4: Transformers, free to watch

Attention

Attention is how a token's representation takes in information from the rest of the context: each token weighs earlier tokens by relevance and mixes in what it finds. Because every token looks at every other, the work grows with the square of the context length.

Taught on Day 4: Transformers, free to watch

Temperature

Temperature sets how strongly a model favors its most likely next tokens. Low temperature gives focused, repeatable output; high temperature spreads probability to less likely tokens and adds variety. Temperature 0 picks the top token each time, but runs still aren't guaranteed to match exactly.

Taught on Day 5: Sampling and reasoning, free to watch

Top-p (nucleus sampling)

Top-p sampling draws the next token only from the smallest set of tokens whose probabilities add up to p, such as 0.9, cutting off the long tail of unlikely choices. Providers advise adjusting top-p or temperature, not both.

Taught on Day 5: Sampling and reasoning, free to watch

Reasoning model

A reasoning model writes out intermediate thinking before its answer, spending extra tokens on hard problems. It is slower and costs more per request, so it pays off on multi-step work such as math, code and planning, and is wasted on simple lookups.

Taught on Day 5: Sampling and reasoning, free to watch

Week 2: Prompts, schemas and tools

System prompt

The system prompt is the instruction block sent ahead of the conversation that sets the model's role, rules and output format for every turn. Models weigh it heavily, but it is not a security boundary: instructions hidden in user input or documents can still override it.

Taught on Day 8: Prompt basics, in Week 2

Few-shot prompting

Few-shot prompting puts a handful of worked examples in the prompt to show the model the task and the format. Models copy examples closely, including their length and their mistakes, so examples should be correct, varied and like real inputs.

Taught on Day 9: Data, format and reasoning, in Week 2

Structured output

Structured output makes a model reply in a fixed shape, usually JSON matching a schema, so code can read it. Providers can enforce the schema while the model writes; your code should still validate the values, because valid JSON can still be wrong.

Taught on Day 10: Structured outputs, in Week 2

Tool calling

Tool calling, or function calling, lets a model ask your code to run a function you described, such as a database lookup, by returning the tool's name and arguments. Your code runs it and sends back the result; the model never executes anything itself.

Taught on Day 11: Tool calling, in Week 2

Rate limit

A rate limit caps how many requests or tokens you can send per minute. Going over returns an error, usually HTTP 429, often with a header saying when to retry. Clients wait and retry with backoff, and spread or batch their work.

Taught on Day 12: Calls that survive, in Week 2

Exponential backoff

Exponential backoff retries a failed call after waits that double each time, such as 1, 2 and 4 seconds, plus random jitter so many clients don't retry together. Retry only errors that can succeed later, such as timeouts, 429 and 5xx, never a bad request, and cap the total time.

Taught on Day 12: Calls that survive, in Week 2

Week 3: Retrieval and RAG: Sourcebook v0.3

RAG (retrieval-augmented generation)

Retrieval-augmented generation answers a question by first searching your documents for relevant passages, then giving them to the model to answer from, with citations. It keeps answers current and grounded in your own data without retraining the model.

Taught on Day 15: The RAG model, in Week 3

Fine-tuning

Fine-tuning continues training a model on your own examples to change its behavior, format or style. It suits consistent outputs on a narrow task, and fits poorly for facts that change: for knowledge, retrieval (RAG) is usually cheaper and easier to keep current.

Taught on Day 15: The RAG model, in Week 3

Chunking

Chunking splits documents into passages small enough to retrieve and fit in a prompt, often a few hundred tokens each, cut along headings and sometimes overlapping. Chunk boundaries decide what can be found: an answer split across two chunks may never be retrieved whole.

Taught on Day 16: Chunk and embed, in Week 3

Embedding

An embedding is a list of numbers that represents a text's meaning, so texts with similar meaning get nearby vectors. Retrieval embeds every chunk once, embeds each question when it arrives, and returns the chunks closest to the question.

Taught on Day 16: Chunk and embed, in Week 3

Vector database

A vector database stores embeddings with their text and metadata and quickly finds the ones nearest a query vector, usually with an approximate index. Metadata filters, such as document type or the user's permissions, narrow the search.

Taught on Day 17: Index, in Week 3

BM25

BM25 is a classic keyword-ranking formula that scores documents by how often the query's words appear, weighted by how rare those words are. It finds exact terms that embeddings blur, such as product codes, names and error messages.

Taught on Day 18: Hybrid search, in Week 3

Reciprocal rank fusion (RRF)

Reciprocal rank fusion merges ranked lists by giving each result 1 / (k + rank) in every list it appears in and summing those scores, with k commonly 60. It uses ranks rather than raw scores, so lists scored on different scales combine cleanly.

Taught on Day 18: Hybrid search, in Week 3

Reranker

A reranker is a second, slower model that reads the query and each retrieved passage together and scores how well they match. Retrieval fetches a wide candidate set cheaply; the reranker reorders the top few dozen so the best passages reach the prompt.

Covered in RAG, Explained

Grounding and citations

A grounded answer says only what its sources support and cites the passage behind each claim. Systems enforce it with numbered sources in the prompt, an instruction to abstain when the sources don't answer, and a check that every citation points to a passage that says it.

Taught on Day 19: Cited answers, in Week 3

Recall@k

Recall@k measures retrieval: the share of a question's relevant passages that appear in the top k results. With two relevant chunks and one in the top 5, recall@5 is 0.5. Check it before answer quality, because the model can't use a passage it never sees.

Taught on Day 21: Smoke eval, in Week 3

Week 4: Agents and tools: a bounded loop

AI agent

An AI agent is a model running in a loop that chooses its own next step: it calls tools, reads the results and continues until the task is done or a limit stops it. Agents suit open-ended tasks; fixed, predictable tasks are cheaper and more reliable as workflows.

Taught on Day 22: Workflows vs agents, in Week 4

Workflow

A workflow chains model calls and code along a path you define in advance, such as classify, then retrieve, then answer. Unlike an agent, the model doesn't choose the steps, so workflows are easier to test, cheaper and more predictable.

Taught on Day 22: Workflows vs agents, in Week 4

Agent loop

The agent loop is the code that runs an agent: send the conversation and tool definitions, run any tool the model asks for, append the result and call again, until the model gives a final answer. Production loops add budgets for steps, tokens and time.

Taught on Day 23: The loop, in Week 4

MCP (Model Context Protocol)

The Model Context Protocol is an open standard for connecting AI applications to tools and data through servers that describe what they offer, so one server works with any compatible client. It doesn't make a server safe: a third-party MCP server runs with your permissions.

Taught on Day 26: Memory and MCP, in Week 4

Week 5: Evaluation and production: Groundtruth

Mean reciprocal rank (MRR)

Mean reciprocal rank averages 1 / rank of the first relevant result across questions: 1 when it comes first, 0.5 when second. It rewards putting the right passage at the top, which matters when only a few passages fit in the prompt.

Taught on Day 31: Retrieval metrics, in Week 5

Error analysis

Error analysis means reading a sample of real outputs, writing down every failure and grouping them into categories before building any metric. It shows which failures are common and worth measuring, instead of guessing.

Taught on Day 29: Error analysis, in Week 5

Golden dataset

A golden dataset is a fixed set of test questions with known answers or labeled relevant passages, used to compare versions of a system fairly. It should cover the real kinds of questions, including ones with no answer, and is best labeled by someone other than the person tuning the system.

Taught on Day 30: Golden dataset, in Week 5

LLM-as-a-judge

LLM-as-a-judge uses a model with a written rubric to grade outputs, such as whether an answer is supported by its sources. It scales evaluation, but its grades are only trustworthy after you compare them with human labels and report how often they agree.

Taught on Day 32: Judging answers, in Week 5

LLM observability

LLM observability records a trace for every request: the prompt, retrieved passages, tool calls, tokens, cost and latency. Traces let you debug one bad answer and watch quality, cost and speed across all of them.

Taught on Day 33: Observability, in Week 5

Prompt caching

Prompt caching lets a provider reuse its work on a repeated prompt prefix, such as a long system prompt or document, across requests. Cached input tokens cost less and return faster, so stable content belongs at the start of the prompt.

Taught on Day 35: Reliability, in Week 5

Week 6: Safety, ML foundations and the ship gate

Prompt injection

Prompt injection is an attack in which text the model reads, such as a web page, an email or a retrieved document, carries instructions that take it over. Prompting alone can't prevent it, so systems limit what the model can do: least-privilege tools, approval for side effects, and retrieved content treated as data.

Taught on Day 36: Threat model, in Week 6

Lethal trifecta

The lethal trifecta, Simon Willison's term, is an AI system that can read private data, takes in untrusted content and can send data out. With all three, a prompt injection can steal data; removing any one leg closes that path.

Taught on Day 36: Threat model, in Week 6

Week 1 teaches the first 11 of these, free, with video lessons.

Start Week 1 free