What is context rot?
Context rot is the tendency of language models to answer less reliably as their input grows, even when the task does not change. A fact the model finds easily in a short prompt gets missed once it sits among tens of thousands of tokens. The term comes from a 2025 Chroma report that measured the effect across 18 models.
Part of Glossary: the terms behind MCP, coding agents and shared pages, defined plainly.
Every token in the input competes for the model's attention. The more there is, the thinner that attention is spread.
Recall drops gradually as length grows, not at a cliff. The effect shows up long before the context window is actually full.
It gets worse when the question and the answer share few words, and when the input holds distractors: passages that look relevant but are not.
Oddly, coherent text hurts more than shuffled text. A well-ordered haystack hides the needle better than a jumbled one.
Agents feel it most. Tool results and file reads pile up, and the instruction from the first turn ends up buried under all of it.
Source: Chroma research: Context Rot
Context rot versus a full context window
Running out of context is a hard failure: the API rejects the request, or the harness compacts. Context rot is the soft failure before that point. The request is accepted, the model answers, and the answer is a little worse than it would have been with a shorter input. Nothing flags it. The same model, asked the same question about the same fact, is simply less reliable when the fact is surrounded by more text.
The term comes from a technical report by Chroma, published in 2025, that tested 18 models from Anthropic, OpenAI, Google and Alibaba. The models differed in how they failed, not in whether they did. Claude models tended to abstain and say no answer could be found; GPT models more often produced a confident wrong answer. All of them got worse as the input grew.
Example: a needle in a long haystack
The classic test hides one sentence (the needle) in a pile of unrelated text (the haystack) and asks a question only the needle answers. Models pass this easily when the needle uses the same words as the question. The report made it harder in realistic ways. When the needle matched the question in meaning but not in wording, accuracy fell faster with length. Adding a single distractor, a passage on the same topic that does not answer the question, lowered scores, and four distractors lowered them further. And a surprise: shuffling the haystack so its sentences no longer formed coherent text made every model do better than on the original, well-ordered version.
The most agent-like experiment used LongMemEval, a benchmark of questions about long chat histories. Given the full history, around 113,000 tokens, models scored far below what they managed when handed only the relevant few hundred tokens. Same facts, same question. The difference was everything else in the window.
Even a trivial task rotted. Asked to copy a list of repeated words with one odd word inserted, every model degraded as the list grew toward 10,000 words.
Why it matters for agents
A coding agent lives in the long-input regime. After an hour, its window is mostly tool results: file contents, test output, directory listings. The instruction you gave at the start is now the needle, and most of what surrounds it is a distractor that looks relevant because it is all about the same codebase. That is exactly the setup the report found hardest. It explains a familiar experience: the agent that was sharp for the first ten minutes starts repeating a mistake you already corrected.
Common mistakes
- Assuming a large window means the problem is solved. The report found degradation well inside every model's limit. Capacity and reliability are different things.
- Adding more context to fix a wrong answer. Often the fix is less: remove the stale tool output, restate the goal, or start a fresh session with a short brief.
- Trusting a confident answer from a long session. Check it against the source, the way you would check a colleague's summary of a long document.
FAQ
- Is context rot the same as running out of context window?
- No. Running out is a hard limit: the request is rejected or the harness compacts. Context rot happens inside the limit, while everything still works, as the quality of answers slips with each extra thousand tokens.
- Which models suffer from context rot?
- Every one of the 18 models Chroma tested, from Anthropic, OpenAI, Google and Alibaba. They differed in how they failed: Claude models more often said no answer could be found, GPT models more often gave a confident wrong one.
- How do I reduce context rot in a coding agent?
- Keep the input focused. Clear or compact old tool output, start a fresh session for an unrelated task, hand large reads to a subagent that returns a summary, and restate the goal rather than relying on an instruction from an hour ago.
Stop sending HTMLover Slack.
Create an account in a minute, connect your coding agent, and make your first board.