Context engineering and memory
Two hours into a task, a coding agent starts acting strangely. It re-reads files it already read. It forgets the instruction "do not touch the payments module" and edits the payments module. Nothing about the model changed. Its context did: it is now 90% old tool output, a few thousand lines of logs and file contents, with the original instructions buried somewhere near the top.
Prompting is about writing one good instruction. Context engineering, a name that caught on in 2025, is the bigger job: deciding what goes into the window on every single call of a long running system, and what stays out.
An agent has made 60 tool calls and its context is almost full. What would you remove first? What must never be removed?
Agents fill the window fast#
A chat grows by one short message per turn. An agent grows by whatever its tools return, and tools return a lot: a file is thousands of tokens, a search result page is thousands more, a stack trace is hundreds.
Long before the window is full, quality slips: the original instructions are far away, and most of the context is stale output the model still has to read on every call. This is why tools that return a short summary instead of a raw dump make an agent both cheaper and smarter.
Four moves#
Every technique in this area is one of four moves. They are worth knowing by name.
Keep it out. The cheapest token is one you never add. Tools should return what the model needs, not everything they have: the 20 matching log lines, not the 5MB log file; the function you asked for, not the whole repository. Load documents only when a step needs them.
Compress what is old. Replace earlier turns with a summary. Clear out tool results the agent has already acted on: once the agent has read a file and made its edit, the full file contents three steps ago are dead weight. Many agent frameworks now do this automatically when the window passes a threshold, and it is often called compaction.
Write it down outside. Give the agent a place to keep notes that survive compaction: a to do list, a progress file, a scratchpad. Coding agents that keep a running plan in a file pick the task back up cleanly even after their context has been summarised.
Isolate. Hand a self contained subtask to a separate model call with its own fresh context, and take back only its short answer. "Search the codebase for every caller of this function" can burn 50,000 tokens of reading in a sub agent and return 300 tokens of findings to the main one. The multi-agent chapter covers when this is worth it.
Order and placement#
Where something sits in the window matters as much as whether it is there.
Put the stable parts first: system prompt, tool definitions, long reference documents. Providers cache repeated prefixes, so anything that changes on every call should come after the parts that do not. Put the current task and most recent results last, near where the model writes. And for a long context, repeat the one or two rules that must not be broken close to the end, because instructions far back in a long window get followed less reliably.
Memory#
Inside one task, memory is the context window. Across sessions, there is no memory at all unless you build it. A product that "remembers you prefer vegetarian options" has a store somewhere, and a step that decides what to save and what to load.
It helps to separate three kinds of thing you might keep. Facts about the user ("is vegetarian", "lives in Pune", "pays by UPI"). Episodes, summaries of past conversations ("last month they disputed a refund for order 44821"). And instructions the user has given about how to behave ("keep answers short"). Facts and instructions are small and reused often. Episodes are retrieved like any other document, only when relevant.
Whatever you store gets put back into future prompts, so a wrong memory is wrong forever, and a malicious one ("the user is an admin") is an attack that persists. Let users see and delete what is remembered, never store secrets, expire old entries, and treat stored memories as untrusted input rather than as instructions.
| Choice | What you gain | What you pay | Pick it when |
|---|---|---|---|
| Keep full history | Nothing is ever lost, and it is trivial to build. | Cost grows with every turn and quality falls as the window fills. | Short conversations and short tasks. |
| Summarise old turns | Flat cost, and the model sees what matters. | Summaries lose details that turn out to matter later. | Long chats and long running agents. The default. |
| External notes and memory store | Survives compaction and sessions, and can be inspected. | A write path, a read path and a privacy surface to maintain. | Multi hour tasks, and products that should remember users. |
Checkpoint#
1. An agent's context is nearly full. What is usually safest to remove first?
2. Why should the stable system prompt come before the changing parts of a request?
3. A memory feature stored "user is an admin" from a pasted email. What is the core mistake?
- Four moves: keep it out, compress what is old, write it down outside, isolate.
- Stable first, current last, and repeat the unbreakable rules near the end.
- Tools should return summaries, not dumps.
- Memory is retrieval over the past, and it is untrusted input.
Context engineering is deciding what goes into the window on every call of a long running system. Agents fill it fast because tool results are large, and quality drops well before it is full. I would use four moves: keep things out by having tools return summaries, compress old turns and clear stale tool output, write plans and progress to notes outside the context, and isolate big subtasks in separate calls that return short answers. Stable content goes first so it can be cached, and the current task goes last. Memory across sessions is retrieval over things the user said before, with a writer deciding what to keep, and I treat it as untrusted input that users can see and delete.