Context Window
The context window is the maximum text — measured in tokens — an LLM can consider at once: prompt, conversation, documents, and its own output combined.
The context window is the token budget for a single request, and everything counts against it: system prompt, conversation history, retrieved documents, tool results, and the response being generated. It stays a budget at any size, because cost scales with tokens, latency grows with input length, and attention dilutes in the middle.
The context window is the maximum number of tokens a language model can process in one request — everything counts against it: the system prompt, conversation history, retrieved documents, tool results, and the response being generated.
It's the defining resource constraint of LLM applications. Frontier models grew from 4K tokens (2023) to 200K as standard with million-token windows on recent Claude models — yet the window stays a budget, for three durable reasons: cost scales with tokens processed, latency grows with input length, and attention dilutes — models recall the start and end of long contexts better than the middle, so the right answer buried under noise often goes unused.
That's why the craft of context engineering — load the relevant slice, not the repo — outlives every window-size increase, why RAG retrieves rather than stuffs, and why agents like Claude Code ship compaction and memory machinery to keep long sessions sharp.
Frequently asked questions
- What happens when the context window fills up?
- Nothing more fits — so something must go. Applications truncate old turns, summarize them (Claude Code's /compact), or retrieve selectively instead of loading everything (RAG). Quality usually degrades before the hard limit: models weight the start and end of a long window more than the middle, so buried facts get missed.
- Bigger context windows keep shipping — does context management still matter?
- Yes. A million-token window changes what's possible (whole codebases, long documents) but not the economics or attention physics: you pay per token processed, latency grows with input size, and signal still competes with noise. A focused window reliably beats a stuffed one — capacity is budget, not license.
Filed under
context · tokens · llm · limits
Related
- Prompt CachingPrompt caching reuses the computed state of a repeated prompt prefix across requests — dramatically cutting cost and time-to-first-token for stable context.
- Token (LLM)A token is the unit LLMs read and write — a word fragment of roughly 3–4 characters in English. Models are priced, limited, and measured in tokens, not words.
- Context EngineeringContext engineering is the discipline of curating exactly what enters an LLM's context window so it has the right information and nothing else.
- Managing Claude Code Memory & Context: CLAUDE.md, /compact, and Auto-MemoryHow Claude Code remembers — every CLAUDE.md scope and load order, path-scoped rules, the auto-memory system, and the context commands that keep sessions sharp.
- LLM Context Windows Compared (2026)Context windows and max output tokens across Claude, GPT, Gemini, DeepSeek, and Grok — the million-token era, what it costs, and what fits in practice.
- Choosing the Right Model: Haiku vs Sonnet vs OpusHow to pick the right Claude model tier — Haiku, Sonnet, or Opus — for any Claude Code agent or task, with a clear decision rubric and per-agent examples.
- RAG (Retrieval-Augmented Generation)RAG retrieves relevant documents from your own data and injects them into an LLM's prompt at query time, grounding answers in facts the model wasn't trained on.
- Token StreamingToken streaming delivers model output incrementally as it's generated — via SSE or websockets — so users see text immediately instead of waiting.