Live on Product Hunt Intraface launched today. Less steering, more vacations. See the launch
Start for free

What is a context window?

A context window is the total amount of text, measured in tokens, that a language model can take into account while producing a reply, including the reply itself. The system prompt, every message so far, tool definitions, tool results and the model's own output all count toward it. When it is full, something has to be dropped or summarised.

Part of Glossary: the terms behind MCP, coding agents and shared pages, defined plainly.

How it works

One request carries instructions, tool definitions, the conversation so far and room for the reply; the reply then becomes history for the next turn.one request, measuredin tokensinstructionssystem prompt, rules filestool definitionsevery tool, every turnconversationmessages, tool resultsthe replythinking and textthe limit is thewhole row togethermodel reads all, writesbecomes history next turn
  1. Every request carries the whole conversation: system prompt, tool definitions, each earlier message and tool result, plus room for the reply.

  2. The model reads all of it as one sequence of tokens. Nothing is kept between requests; what is not in the request does not exist to the model.

  3. Each reply, including any thinking, is appended and sent back as input on the next turn, so the window fills a little faster every step.

  4. Long tool results, pasted files and images are the usual culprits. One large file read can cost more than an hour of ordinary chat.

  5. Near the limit the harness has to act: summarise the history (compaction), clear old tool results, or start fresh. Past the limit the API refuses the request.

Source: Anthropic docs: Context windows

Context window versus memory

People say a model "remembers" the conversation. It does not. Every request to the API carries the entire history again, and the model reads it from the top as one long sequence of tokens. The context window is the ceiling on that sequence. Memory, in the sense of something that survives between sessions, has to live somewhere else: a file on disk, a database, a note the agent writes to itself. The window is working memory. It is emptied when the session ends, and it is full long before most people expect.

Two details matter. First, the model's own output counts. If it thinks before answering, those thinking tokens take space too, and whether earlier thinking blocks stay in later turns depends on the model. Second, prompt caching does not free anything. A cached prefix is cheaper to process, but it still occupies the same tokens.

Example: one coding session

Open a coding agent in a repository. Before you type anything, the window already holds the system prompt, your instruction files, the definitions of every tool the agent can call, and the names of the MCP servers you have connected. Ask it to fix a bug. It reads four files, and each lands in the window in full. It runs the test suite, and the output lands too, including the 300 lines of passing tests you never look at. Twenty turns later the window is mostly tool output, and the sentence where you said not to touch the migration files is a tiny fraction of what the model is attending to.

That is why agents compact. Claude Code summarises the conversation when the window gets close to full and carries on from the summary, keeping the decisions and dropping the noise.

Why it matters

Three things get worse as the window fills. Cost: every token is billed on every turn, so a long conversation pays for its history again and again. Latency: more input means a slower first token. Accuracy: recall degrades as the input grows, a pattern known as context rot. A bigger window raises the ceiling on the first two and does nothing for the third. The practical rule is the opposite of what the limit suggests: treat the space as scarce even when plenty is left.

Common mistakes

  • Pasting whole files when a path would do. Agents can read a file when they need it. Pre-loading everything spends the budget before the task starts.
  • Letting tool output pile up. A test run, a log tail, a directory listing. Each is useful once, then dead weight. Clear between unrelated tasks, or hand large reads to a subagent whose window is separate from yours.
  • Reading the limit as a target. The error for exceeding the window is obvious. The slow slide in quality beforehand is not.
  • Forgetting that tools count. Ten connected MCP servers add every tool's description to every request, before the first message.

FAQ

Is a larger context window always better?
No. A larger window lets you send more, but recall gets worse as the input grows, so a model given everything often answers less accurately than one given the relevant part. Treat the space as a budget, not a target.
What happens when the context window is full?
If the input alone is over the limit, the API rejects the request with an error. Agent harnesses avoid that by compacting: Claude Code summarises the conversation as it nears the limit and continues from the summary.
Do tool definitions count toward the context window?
Yes. Every tool the model could call is described in the request, on every turn, before a single message. Connecting several MCP servers with many tools each can take a noticeable share of the window before the work starts.

Stop sending HTMLover Slack.

Create an account in a minute, connect your coding agent, and make your first board.