Context Engineering

Prompt engineering is about the one message you craft for a single interaction. Context engineering is about everything the model sees at every step of a long agent run — and in 2026, that distinction is the whole ballgame.

The term was popularized by Shopify CEO Tobi Lütke on June 19, 2025, who framed it as "the art of providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy endorsed it as "the delicate art and science of filling the context window with just the right information for the next step."

The scope is the key difference. A prompt is one string. A context is the entire state assembled before each model call: the system instructions, the running conversation, retrieved documents, tool definitions, prior tool results, and long-term memory. In an agent that loops 80 times, you are not writing one prompt — you are managing how that whole pile evolves, grows, and decays across every iteration.

This is why the practitioner consensus in 2025–2026 has broadly shifted: prompt engineering alone is no longer sufficient for production AI. The prompt is now just one ingredient in a much larger, dynamic context that you must engineer deliberately.

It is tempting to think a 200K-token window means you have 200K tokens of usable working memory. You don't. Performance degrades well before you hit the limit, and that degradation is universal.

This is context rot: measurable accuracy loss as input length grows. Chroma's 2025 study of 18 frontier models — including GPT-4.1, Claude Opus 4, and Gemini 2.5 Pro — found degradation at every length increment. It is architectural, rooted in the quadratic cost of attention, not a gap that more training closes. A bigger window does not fix it.

The most famous symptom is the lost-in-the-middle effect (Liu et al., Stanford/TACL 2024): information buried in the middle of a long context is often ignored, with accuracy drops exceeding 30% versus the same fact placed at the start or end.

The practical consequence reframes everything: the context window is a budget shared by competing claimants — instructions, history, tool outputs, retrieved chunks. Every token you spend on low-signal content is a token taxed from the model's attention. High-signal minimalism beats information-dumping. The job is not to fill the window; it is to fill it well.

Drew Breunig's widely-cited June 2025 taxonomy names four distinct ways a context degrades an agent — each with a different mechanism and fix.

1. Context poisoning. A hallucination enters the context, gets written back, and is then treated as ground truth in every subsequent step — compounding the error. Note this is not the same as a one-off hallucination; the danger is the feedback loop that amplifies it.
2. Context distraction. As history accumulates, the model over-relies on its past steps instead of reasoning freshly, repeating prior actions rather than adapting.
3. Context confusion. Irrelevant content — too many tool definitions, stale chunks — influences the response, e.g. the model calls the wrong tool because a tempting-but-wrong one is in scope.
4. Context clash. Two pieces of contradictory information sit in the same window (an old plan and a revised one), producing inconsistent, conflicting guidance.

The lesson: failures are not random. Each maps to a specific corrective strategy — pruning poisoned state, summarizing to reduce distraction, scoping tools to reduce confusion, and reconciling or removing contradictions to resolve clash.

If the window is a budget, these are the four moves you make to stay under it. Think of them as the levers on a mixing board: you push some content out of the window, pull only what you need back in, shrink what remains, and wall off the messy parts. LangChain formalized this production toolkit into four named strategies — treat them as a checklist for any long-running agent.

The common mistake is equating context engineering with RAG alone. RAG is only the select strategy. A robust agent uses all four: it writes intermediate results to a scratchpad so they don't bloat the window, selects the few documents it needs right now, compresses old turns into summaries, and isolates risky sub-tasks into clean sub-agent contexts.

A second high-leverage technique cuts across all four: static/dynamic separation. Put rarely-changing system instructions first so they can be prompt-cached for cost and latency, and put dynamic content (new tool results, fresh retrievals) last to exploit the model's recency bias. Caching complements context engineering; it does not replace the curation work.

Two patterns do most of the heavy lifting in production. Both follow the same instinct a careful researcher uses: keep your desk clear, and pull a reference only when you actually open it.

Just-in-time retrieval. Instead of pre-loading every possibly-relevant document, the agent holds lightweight identifiers — file paths, stored queries, URLs — and loads the actual content into context only when a step needs it. Think of it as keeping a table of contents in mind rather than the whole library on the desk. This progressive disclosure keeps the active context small and high-signal.

# Don't pre-stuff everything. Hold references; load on demand.
file_index = ["docs/auth.md", "docs/billing.md", "docs/api.md"]

def step(state):
    # Agent decides which reference it needs *now*
    needed = model.pick_relevant(state.goal, file_index)
    content = read_file(needed)            # load into context just-in-time
    return model.act(state.goal, content)  # small, focused context

Compaction. When the conversation grows past a threshold, summarize it — preserving high-signal artifacts (architectural decisions, unresolved bugs, key identifiers) and discarding low-signal content (redundant tool dumps, already-resolved chatter). Anthropic's Claude API ships a native compaction API (beta compact-2026-01-12, type compact_20260112) that auto-summarizes when input tokens exceed a configurable threshold (default 100,000; minimum 50,000). Claude Code triggers auto-compaction at roughly 95% of context capacity. Crucially, compaction is not lossless — the art is choosing what to keep. The Claude Cookbook documents the three core levers: compaction, tool-result clearing, and memory injection.

The strongest lever against context failure is structural: don't let one context window absorb everything.

In sub-agent isolation, a main orchestrator spawns ephemeral sub-agents, each with a clean, task-scoped context window. A sub-agent does its messy work — dozens of tool calls, dead ends, verbose outputs — in its own context, then returns a condensed summary (typically 1,000–2,000 tokens) to the parent. The parent never sees the noise; the sub-agent always starts clean. Anthropic measured a 90.2% performance improvement for this architecture over single long-context agents.

def orchestrator(goal):
    subtasks = plan(goal)
    summaries = []
    for task in subtasks:
        # Each sub-agent gets a fresh, isolated context
        result = run_subagent(task)          # noisy work stays inside
        summaries.append(result.summary)     # only ~1–2k tokens returns
    return synthesize(goal, summaries)

But isolation redistributes context management rather than eliminating it: the orchestrator still accumulates summary traffic and must manage its own growing context. Active research continues to target the residual problem of cross-agent context contamination when many sub-agents share a single orchestrator's window. Isolation is powerful, not free.