What an agent is
"Agent" is the most stretched word in software right now. It gets used for a chatbot, a cron job with a prompt in it, and a system that writes and ships code on its own. A definition you can actually design with: an agent is a model that decides its own next step, in a loop, using tools, until it judges the task done.
The important words are "decides its own next step". If your code decides the steps and the model only fills them in, that is not an agent. It is a workflow, and most of the time a workflow is what you should build.
A feature must read a complaint, look up the order, and decide whether to refund. Does it need an agent? What would you build instead, and why?
Workflows and agents#
Anthropic drew this line in a widely read 2024 guide, and it is the most useful distinction in the whole area. Flip between the two.
The loop#
Every agent, from a support bot to a coding assistant, is the same loop. Here it is handling a request no fixed workflow anticipated.
The loop stops when the model answers instead of calling a tool. In production it also needs stops the model cannot talk its way past: a maximum number of steps, a token or money budget, a timeout, and a human approval in front of anything irreversible. Add a check for repetition too, because a confused agent will happily call the same failing tool twenty times.
Why agents fail: errors compound#
This is the number that should shape every agent you build. Every step can go wrong, a wrong tool, a misread result, a bad argument, and the task only succeeds if every step goes right.
A step that is right 95% of the time sounds excellent. Chain ten of them and four tasks in ten fail. This is why good agents are built from fewer, more reliable steps, and why they check their own work (run the tests, re-read the result) so a bad step gets caught and retried rather than carried forward.
Three things fight compounding. Fewer steps: better tools that do more per call. Checks inside the loop: a coding agent that runs the tests after every change finds its own mistakes. Recovery: clear error messages from tools, so the model can try again differently instead of guessing.
When to build an agent#
Start with the simplest thing that could work, and only climb when it fails:
- One model call, with good retrieval and a good prompt. This covers more than people expect.
- A workflow: several calls in an order your code fixes.
- An agent: only when the steps genuinely cannot be known in advance.
An agent earns its cost when the task is open ended (debug this, research that, sort out this customer's account), when a wrong step can be detected and undone, and when the value of finishing the task is higher than the cost of the extra calls.
| Choice | What you gain | What you pay | Pick it when |
|---|---|---|---|
| Single model call | Fastest, cheapest and easiest to test. | Cannot fetch anything or take multiple steps. | Classification, extraction, answering from provided context. |
| Workflow | Predictable path, testable step by step, cost known in advance. | Breaks on cases the fixed path did not anticipate. | Most production features. The steps are known. |
| Agent | Handles open ended tasks and surprises. | Many calls, variable cost and latency, and errors that compound. | The steps depend on what you find along the way, and mistakes can be caught. |
An agent can only do what its tools allow. Read only tools make it an assistant. A tool that sends email or moves money makes it something that can cause an incident at 3am. Start with read only tools, add write tools one at a time with limits in code, and put approval in front of anything you cannot undo.
Checkpoint#
1. What makes a system an agent rather than a workflow?
2. Each step of your agent is right 95% of the time. A typical task takes 14 steps. Roughly how often does a task go fully right?
3. Which stop condition must never depend on the model deciding to stop?
- Agent: the model picks its next step in a loop. Workflow: your code does.
- Reliability multiplies. Ten steps at 95% is about 60%.
- Hard stops in code: max steps, budget, timeout, approval for the irreversible.
- Climb only when needed: one call, then a workflow, then an agent.
An agent is a model that chooses its own next step in a loop, calling tools that my code runs, until it decides the task is done. If my code fixes the steps, it is a workflow, and that is what I would build whenever the steps are known, because it is cheaper, predictable and testable. The loop sends the goal and everything so far on each turn, since the model has no memory, and it needs hard limits in code: max steps, a budget, a timeout and approval before anything irreversible. The key number is that reliability multiplies, so ten steps at 95 percent each succeed only about 60 percent of the time, which is why good agents use fewer, stronger tools and check their own work.