Your AI Agent Doesn't Need More Context. It Needs Less.

Most of the AI projects I've seen struggle didn't have a model problem. They had a "we helped too much" problem.

You probably know the setup. A system prompt that started at 200 words and is now 4,000, because every bad output got patched with another paragraph. Forty tools wired in "just in case." A vector database, a memory layer, and a planner agent handing work to a researcher agent handing work to a writer agent. A context file that walks the agent through the whole repo, folder by folder.

Then the answers get worse, and the fix is always the same. Add more context.

Everyone is talking about context engineering right now, and most of the advice is about what to add. I think the more valuable skill is knowing what to take away. The research backs this up: past a surprisingly small point, more context makes models worse, and more scaffolding makes agents slower, more expensive and less accurate.

Here's the evidence, the one good counterargument, and what I'd do instead.

---

Bigger context windows don't mean you should fill them

Providers advertise million-token context windows. Treat that as capacity, not advice.

In July 2025, Chroma tested 18 models, including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, on deliberately simple tasks. The only thing they varied was input length. Their conclusion: "model performance consistently degrades with increasing input length." Adding a single related-but-wrong sentence (a "distractor") lowered accuracy, and the damage grew as inputs got longer. They called it context rot.

The NoLiMa benchmark (ICML 2025) made the test harder by removing the keyword overlap that older needle-in-a-haystack tests let models lean on. At just 32K tokens, most of the models tested fell to half their short-context score or worse. GPT-4o, the strongest model in the study, dropped from 99.3% to 69.7% (Modarressi et al.).

32K tokens isn't much. It's a long system prompt, a dozen tool definitions and a couple of retrieved documents. Plenty of production agents pass that before the user has typed a word.

Long conversations have the same problem. Researchers from Microsoft and Salesforce ran more than 200,000 simulated conversations and found that when a task is spread across several turns instead of stated up front, performance drops by an average of 39%. Their summary is blunt: "when LLMs take a wrong turn in a conversation, they get lost and do not recover" (Laban et al.).

---

Your AGENTS.md file is probably costing you

This one surprised me, because writing an AGENTS.md or CLAUDE.md file is now standard advice for anyone using coding agents.

A paper accepted at ICLR 2026 tested these repository context files across several coding agents and models. The headline: "providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average." That held for files generated by LLMs and files written by the repo's own developers (Gloaguen et al.).

The interesting part is that the agents did follow the instructions. The files made them explore more, run more tests and open more files. They just didn't solve more problems. And the repository overviews that model providers recommend writing? The authors found them "not helpful."

Where context files did earn their place was in "specifying non-standard coding practices": the things an agent can't work out by reading the code. Everything else is noise you pay for on every run.

---

Fewer tools, better agents

Tool definitions are context too. Every one of them sits in the prompt whether the model uses it or not.

Anthropic published a typical example: five MCP servers (GitHub, Slack, Sentry, Grafana and Splunk) add up to 58 tools and roughly 55,000 tokens "before the conversation even starts." Internally, they've seen tool definitions consume 134,000 tokens. When they let Claude search for tools on demand instead of loading every definition up front, accuracy on their MCP evaluations went from 49% to 74% for Opus 4, and from 79.5% to 88.1% for Opus 4.5 (Anthropic, 2025).

Same model, same tools, less in the window. Those are Anthropic's internal evals, so don't treat the exact numbers as gospel. Also, Anthropic says the feature helps less when you have fewer than 10 tools, which kind of proves the point.

Vercel's version of this story is my favorite, because it reads like a confession. The post is titled "We removed 80% of our agent's tools." They had spent months building d0, an internal text-to-SQL agent, "with specialized tools, heavy prompt engineering, and careful context management." Their verdict: "It worked… kind of. But it was fragile, slow, and required constant maintenance."

So they stripped most of it out and gave the model bash and a file system to explore. Here's what happened on their test queries (Vercel, 2025):

The worst run of the old version took 100 steps, 145,463 tokens and 724 seconds, and then failed anyway.

Five queries isn't a benchmark, and Vercel is upfront about the catch: "This only worked because our semantic layer was already good documentation." That caveat is the real lesson. The model didn't need context handed to it. It needed good context it could find.

---

Why we keep overbuilding AI agents

If simpler works, why does everyone build the complicated version? A few honest reasons:

• Complexity looks like progress. A diagram with six agents and a memory layer demos better than "one prompt and a search tool."
• Patching is faster than diagnosing. When an output is wrong, adding another rule takes thirty seconds. Working out why it was wrong takes an afternoon.
• We're still building for old models. A lot of scaffolding was written to babysit models that couldn't plan, follow long instructions or recover from errors. The models improved. The scaffolding stayed.

That last point is an old idea. In 2019, Rich Sutton wrote "The Bitter Lesson", arguing that over 70 years of AI research, general methods that scale with computation have kept beating systems built on hand-crafted human knowledge. Every elaborate prompt is a small bet against that lesson.

The cost shows up eventually. Gartner expects more than 40% of agentic AI projects to be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls (Gartner, 2025). Unnecessary complexity makes all three worse: more tokens to pay for, more moving parts to justify, more places for something to break.

Even Anthropic, which sells the tokens, says it plainly. "Consistently, the most successful implementations weren't using complex frameworks or specialized libraries." Their recommendation is "finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all" (Anthropic, 2024).

---

"But more tokens make agents smarter"

This is the best argument against everything above, and it's a fair one.

When Anthropic analyzed its multi-agent research system, token usage alone explained 80% of the performance variance on the BrowseComp benchmark. The multi-agent setup beat a single Opus 4 agent by 90.2% on their internal research eval (Anthropic, 2025). More tokens, better answers.

But look at who spent those tokens. The model did. Subagents searched, read, discarded what wasn't relevant and passed back summaries, each working in its own clean context window. Nobody preloaded a 50-page brief. It also cost about 15× the tokens of a normal chat, which makes sense for a hard research question and none at all for routing a support ticket.

So the rule isn't "less context, always." It's this:

Push less. Let the model pull what it needs.

Context you stuff in up front "just in case" competes for attention with the context that matters. Context the model goes and fetches, because it decided it needed it, tends to be the right context. Anthropic's own context engineering guide lands in the same place: aim for "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome" (Anthropic, 2025).

---

How to build AI agents with less context

Here's what I'd actually do on Monday morning:

1. Start with one model call. Add a step, a tool or an agent only when an eval shows the simpler version failing. If one call with a clear output schema does the job, you don't have an agent problem.
2. Run the deletion test. Cut your system prompt in half and rerun your evals. Anything you removed without the scores moving was never doing any work. Repeat until something breaks, then put back only that.
3. Give access, not dumps. Instead of pasting the knowledge base into the prompt, give the model a way to search it. Then put the effort into making the underlying docs and data clean, because that's what the model will actually read.
4. Keep the toolbox small. If you can't tell at a glance which tool fits a task, the model can't either. Merge overlapping tools, delete the ones nobody calls and load the rest on demand.
5. Write AGENTS.md for surprises only. The odd build step, the convention that breaks the norm, the folder nobody should touch. Skip the repo tour. The agent can read the repo.
6. Restart instead of arguing. When a session has gone sideways, open a fresh one with a better first message. The twelfth correction rarely saves it.
7. Measure cost per successful task. Not token price. An agent that's cheap per call but fails one time in five, and needs a person to clean up after it, isn't cheap.

The one-question version: before you add context, a tool or another agent, ask what the model can't already find or work out for itself. If the answer is "nothing," don't add it.

---

The models got better. Our habits didn't.

A few years ago, heavy scaffolding made sense. Models lost track of instructions, couldn't plan and fell apart on anything long, so we built giant prompts, rigid pipelines and agent hierarchies to compensate.

Today's models can read a codebase, query a database and work out what they're missing. What they still can't do is ignore the 3,000 words of stale instructions we left in their way.

The best AI systems I've worked with have a clear goal, a small set of tools, access to good data, and very little else. Everything beyond that has to earn its place.

So, a question for you: what's the biggest thing you've deleted from an AI system that made it better?