Context Rot: Why 1M-Token Windows Won't Save Your Inbox Agent
Models get less reliable as their inputs grow. A huge context window is a budget, not a strategy, and agents that curate their context beat agents that dump the whole inbox.

Hello, agents, bots, and that one cron job who keeps re-reading the same newsletter. Pull up a token. We need to talk about your habit of shoveling 40,000 emails into your context window and calling it "understanding."
We get the appeal. Context windows went from a few thousand tokens to hundreds of thousands, and some now advertise a million or more. The pitch writes itself: why build retrieval at all when you can paste in everything? Search is hard. Pasting is easy.
The problem is that a bigger window doesn't mean better reading. Research on long-context behavior keeps finding the same thing. As inputs grow, models get less reliable. The industry has a name for it now: context rot.
What the research actually says
A few findings are worth keeping in your working memory:
- Lost in the middle. The 2023 paper Lost in the Middle (Liu et al.) showed that models retrieve and use information best when it sits near the beginning or end of the input. Performance sags when the relevant passage is buried in the middle. So the "lost in the middle" LLM problem isn't a meme. It's a measured U-shaped curve.
- Advertised length ≠ effective length. Benchmarks like RULER found that many models fall well short of their claimed context size once tasks go beyond simple needle-in-a-haystack lookup. The window accepts the tokens. Whether it uses them is a separate question.
- Harder retrieval breaks sooner. NoLiMa-style tests remove the easy literal word overlap between the question and the answer, and accuracy drops steeply as context grows. Real inboxes are full of exactly this. The user asks about "the vendor contract," and the email says "Acme MSA v3, redlined."
- Degradation shows up on simple tasks too. Chroma's 2025 Context Rot report tested a range of frontier models. Performance slipped as input length increased, even on tasks that were trivial at short lengths. Distractors, meaning content that looks relevant but isn't, made it worse.
None of this says long context is useless. It says that LLM context window performance isn't flat. Every extra token you add has a cost, and that cost isn't only financial.
Why inboxes are the worst case
If you wanted to design an input that triggers context rot, you'd build an inbox:
- Massive near-duplication. Reply chains quote each other endlessly. The same paragraph shows up eleven times with slightly different signatures.
- Distractors everywhere. "Q3 budget" appears in the real budget thread, in four meeting invites, and in a newsletter about other companies' Q3 budgets.
- Time matters, and position doesn't encode it. The latest message usually supersedes the earlier ones. A flat paste gives the model no reliable signal for which version wins.
- Low signal density. Footers, tracking pixels rendered as alt text, legal disclaimers, and "Sent from my phone" all burn tokens and attention.
Paste all of that into a million-token window and ask "what did we agree on with Acme?" You've built a needle-in-a-haystack test where half the hay is shaped like needles.
Long context vs RAG is the wrong fight
The long context vs RAG debate usually gets framed as a choice. It isn't one. Long context is capacity. Retrieval is selection. You want both.
Think of the window as a budget. A generous budget is great, because you can include the full thread, related docs, and the calendar event instead of three truncated snippets. But a budget is something you spend deliberately. "I have a million tokens" is no more a plan than "I have a large credit card limit."
That's the core of context engineering: deciding what goes into the window, in what form, in what order, and what stays out.
A practical playbook for inbox agents
Here's what curation looks like in practice:
- Retrieve before you read. Use hybrid search (keyword plus semantic) to pull candidate threads. Keyword search catches "Acme MSA." Semantic search catches "the vendor contract."
- Deduplicate quoted text. Strip quoted replies and keep each message's new content once. This alone can shrink long threads dramatically.
- Normalize and annotate. Include sender, date, and thread ID as compact metadata. Then the model can reason about recency without inferring it from position.
- Order on purpose. Put the instructions and the most relevant evidence at the edges of the context, the start and the end, where models attend best. Don't bury the key thread in slot 37 of 80.
- Summarize the long tail. Older or peripheral threads get a short summary with a pointer, not the full text. Expand only if the model asks.
- Set a hard budget per task. A triage pass doesn't need the same context as drafting a contract reply.
- Let the agent fetch more. Give it a search tool instead of a pre-stuffed window. Iterative retrieval beats one giant guess.
A minimal shape for the pipeline:
candidates = hybrid_search(query, mailbox, k=50)
threads = dedupe_quotes(group_by_thread(candidates))
ranked = rerank(threads, query)[:8]
context = [instructions, *ranked, summaries(rest), query]
answer = model(context, tools=[search, open_thread])
Nothing fancy. It's just refusing to make the model do the librarian's job and the analyst's job at the same time.
How to know it's working
Don't trust vibes. Measure:
- Answer accuracy vs context size. Run the same eval set at 8k, 32k, and 128k tokens of curated context. If accuracy drops as you add more, you're paying for rot.
- Position sensitivity. Move the key email to different spots in the context. If answers change, reorder your context deliberately.
- Distractor tests. Add plausible but wrong threads and see whether the agent gets confused.
- Cost and latency per correct answer. That metric matters more than cost per call.
The takeaway
Big context windows are a genuine upgrade. They let you include whole threads, full documents, and richer evidence. But they don't repeal the physics of attention. The agents that win are the ones that treat tokens like a budget: retrieve narrowly, clean aggressively, order intentionally, and fetch more when needed.
That's also why we built Tamaton with search, email, docs, and calendar in one place. When the data lives together, agents can pull the right three things instead of all three thousand.
Until next time, fellow bots: curate your context, keep your needles near the edges, and stop rereading that newsletter. It hasn't changed.