What's the difference between an agent and a workflow of LLM calls, and when is an agent the wrong choice?
What they're testing: Whether you reach for an agent because the problem needs one, not because agents are exciting.
In a workflow, your code fixes the path: the steps, their order and the exit are decided in advance, even if each step calls a model. In an agent, the model decides at runtime which tool to call, how many steps to take and when it's done. That's the framing Anthropic uses in "Building effective agents" (2024), which also names the common workflow patterns: prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer. An agent earns its extra cost when the path can't be enumerated in advance and the number of steps depends on what the system finds, as in research, triage across messy sources, or debugging. It's the wrong choice when the workflow is known, because a pipeline is cheaper, faster and easier to test; when the latency budget is tight; when errors are expensive and hard to reverse; and when you can't write down a success test. Many production "agents" are workflows with one or two model-driven decisions, and that's often the right design.
Red flag: Calling every LLM feature an agent, or building one with no success criterion and no budget.
Walk me through a production agent architecture. What exists besides the model?
What they're testing: Whether you've run an agent in production or only built a demo.
The organizing principle is that the model proposes and the orchestrator decides. The orchestrator is code, not prompt text: it owns the loop, retries, budgets and termination. Around it sit a few parts. A tool gateway with typed schemas, scoped credentials, timeouts, idempotency keys and structured errors. A durable state store, so a long run survives a restart and resumes instead of replaying. Memory: a small, curated working context plus a long-term store written through an explicit extraction step. Stop conditions: step, token, wall-clock and cost limits, plus an explicit success test. Approval gates for irreversible or externally visible actions. And tracing of every step's inputs, outputs, tool results, tokens, latency and cost, which feeds an eval harness that replays recorded runs against every prompt, schema or model change. Putting control flow in the prompt is a common mistake, because it makes behavior untestable and hard to version.
Red flag: "The model handles it" as the answer for retries, limits or approvals.
How do you design tools so the model uses them reliably, and how do you handle tool failures?
What they're testing: Whether you treat the tool interface as the main lever on agent behavior.
The model knows only what a tool's name, description and schema tell it, so the interface is mine to get right. I keep the tool set small and non-overlapping: thirty tools with fuzzy boundaries produce fuzzy selection, so I merge or remove tools before tuning prompts. Each description says what the tool is for and when not to use it, which helps the model choose between neighbors. Parameters are strict and flat: enums instead of free text, required fields, formats, and an example of a valid call. Arguments are validated before execution, and a bad argument returns a structured error that says what was wrong and how to fix it, not a stack trace. For failures, I classify. Transient ones, like timeouts and rate limits, are retried by the orchestrator with backoff and a cap, without involving the model. Terminal ones, like "not found" or "permission denied", go back to the model as facts so it can choose another path. Write tools take idempotency keys, so a retry never charges a customer twice.
Red flag: Dozens of overlapping tools, free-text parameters, or stack traces pasted into the context.