You paste a 20-page contract into your AI assistant. It answers the first few questions perfectly. Then you ask about a clause from page three. "I don't see that in what you've shared." But you did share it. You pasted the whole document.
This is not the AI lying to you. It is the AI running out of whiteboard space.
To understand why this happens, you need to know about two concepts: tokens and context windows. Neither is complicated, but until you understand them, AI tools will keep surprising you in ways that feel like bugs. They are not bugs. They are physics.
You type words. The AI sees tokens.
When you type into an AI assistant, you see a sentence. The model does not. It sees a sequence of numbers, each one representing a small chunk of text called a token.
A token is not a word. It is more like a syllable or a meaningful fragment of text. Common short words like "the," "is," and "a" are usually one token each. The word "support" might be one token or two depending on the model. The word "tokenization" can split into three or four pieces. Punctuation often gets its own token. Spaces sometimes attach to the following word rather than standing alone.
As a rough guide: 100 tokens is roughly 75 words of ordinary English prose. A typical email runs to a few hundred tokens. A 10-page PDF lands somewhere between 5,000 and 8,000 tokens. A full software codebase can be millions of tokens.
Why does this matter? Because the AI does not process meaning the way you do, reading left to right, weighing each sentence. It processes a fixed number of these numerical chunks at once. That fixed number is the context window.
The context window: a whiteboard you share with the AI
Think of the context window as a physical whiteboard in a shared meeting room. Every token you feed the AI writes something on that whiteboard. Your question goes on there. The AI's answer goes on there. The follow-up message, the clarification, the paste of that contract, all of it goes on that one shared surface.
The whiteboard has a fixed size. Current models can hold anywhere from tens of thousands to several hundred thousand tokens in context, and the number keeps growing with each generation. That sounds enormous. But here is the thing: the whiteboard fills up faster than you expect when you are doing serious work.
Paste a long legal document: a big chunk of the whiteboard is gone. Work through a 60-message brainstorm: the early messages are starting to crowd the later ones. Ask the model to review a long codebase file by file: you are spending tokens with every exchange.
The context window holds everything: your input, the model's output, and every prior turn in the conversation. It is one shared pool, not separate slots for "what I said" and "what it said."
What happens when the whiteboard fills up
Most AI tools handle a full context window in one of two ways.
The first is a hard stop: the tool refuses to accept more input and tells you the conversation has exceeded its limit. You have probably seen this. It usually says something about a context length or a conversation being too long.
The second, which many modern tools use quietly, is a sliding window: as the whiteboard fills, older content gets erased from one end to make room for new content at the other. The model does not "forget" the way a person forgets something after a distraction. It literally no longer has access to the erased tokens. They are gone from the whiteboard. When the AI says "I don't see that in what you've shared," it means exactly that: the relevant passage has been pushed off the edge.
This is why pasting a very long document at the start of a conversation is riskier than it looks. You are using up most of your whiteboard on the document itself, leaving little room for the back-and-forth that follows. The conversation can appear to be working fine right up until an earlier section quietly disappears.
Why speed and cost are also token problems
If you have ever used an AI API directly, rather than a chat interface, you have seen that pricing is listed per token: so many dollars per million input tokens, a different rate per million output tokens.
This is not arbitrary pricing. The model does real computational work for each token it processes and each token it generates. Longer inputs cost more to process. Longer outputs cost more to produce. A prompt that includes thousands of tokens of background context costs more than a focused, tight question covering the same ground.
Response speed follows the same math. "Tokens per second" is how model speed is benchmarked, not words per minute or pages per hour. A model generating a long response is producing tokens one by one. The more tokens it needs to write, the longer the wait. This is why concise, well-targeted prompts tend to get faster, cheaper answers than verbose ones.
Four things you can do differently
Understanding tokens changes how you work with AI tools. None of this requires knowing how models are built. You just need to think about the whiteboard.
Summarize rather than paste in full. Instead of pasting a 50-page document and asking questions, paste a summary you wrote yourself, or paste only the sections directly relevant to your question. You get the same result and preserve whiteboard space for the conversation that matters.
Start a new chat for a new topic. Each conversation begins with an empty whiteboard. If you have been working on one project across 40 messages, switching to a completely different task in the same chat means carrying 40 messages of irrelevant context. Open a new conversation instead.
Put the most important instructions at the top. Models tend to weight context near the beginning and near the end of the whiteboard more reliably than content buried in the middle. If you have a critical constraint or format requirement, it belongs in the first few lines of your prompt, not in a parenthetical three paragraphs down.
Watch the context indicator. Most AI interfaces now show a token counter or a usage bar. Treat it the way you treat a fuel gauge. When you are close to full, wrap up the current thread or start a fresh one rather than waiting for a hard cutoff or a silent erasure.
The invisible problem is the real problem
The most consequential context window issues are the ones you never see. The AI does not announce when it has quietly dropped earlier content from its working memory. It just stops referencing it. Your question gets a partial answer. A constraint you gave earlier gets ignored. A summary misses a section that fell off the whiteboard ten messages ago.
Knowing that tokens and context windows exist lets you design your prompts around those limits rather than stumbling into them. It explains why AI tools work better on focused, bounded tasks than on sprawling, ever-growing conversations. It explains why breaking a large task into smaller, separate sessions often produces better results than trying to do everything in one long thread.
The whiteboard is powerful. It is also finite. The people who get the most out of AI tools are the ones who learn how big it is and stay conscious of how fast they are filling it.
Questions, pushback, or a use case I missed? Get in touch.