Everyone has had the experience. You paste a document into a chat window, ask a sharp question, and get an answer that looks like it came from a very smart colleague. So you paste the next document. And the next. And somewhere around the fifth document, the answers start getting worse. It repeats a point you made three messages ago. It forgets a number it gave you correctly twenty minutes earlier. It starts agreeing with whatever you last said.
The model didn't get stupider. It ran out of room.
The window is smaller than you think
Large language models read their input through a "context window": a fixed budget of tokens, where a token is roughly a word. A modern window is big: 128k or 200k tokens is common, which sounds enormous. In practice it's not. A typical engagement document set (financials, contracts, prior-year notes, regulator guidance) runs to hundreds of pages. A single page is roughly 500 tokens. Two hundred pages is a hundred thousand tokens before you've written a single question.
So the first thing that happens on a real project is that the window fills. What does the model do when it's full? It doesn't refuse. It compresses: it keeps the most recent part of the conversation and quietly forgets the oldest part. If the thing you needed it to remember was at the start (the client's actual constraint, the one number everything hinges on) that's gone.
The forgetting is worse than people think
There's a well-documented effect researchers call "lost in the middle." Give a model a long context and it handles the beginning well, handles the end well, and is noticeably worse on the material in the middle: precisely where the details of a long project live. It's not that the model is lazy. It's that attention is finite and the model spends it unevenly. The middle of a long engagement document is exactly where the facts get dropped.
Add to that the way long chats degrade. Each turn adds to the context. After twenty or thirty exchanges, the model is spending its attention re-processing everything before it, and accuracy drifts. Studies of long conversations show models become more repetitive and more prone to errors as the thread grows: not because the task got harder, but because the working memory got more crowded.
Statelessness is the deeper problem
Here's the uncomfortable part: even with an infinite window, a chat model isn't a system of record. Each message is a fresh pass over the text in front of it. The model doesn't have a persistent understanding of "this engagement": it has whatever it can see in the current window. Close the tab and the project is gone. Come back tomorrow and you're starting from a summary someone wrote, which is itself lossy.
This matters enormously for consulting work, because consulting is cumulative. A strategy storyline is built from a hundred earlier decisions: the market chapter sets up the competitive chapter, which sets up the financial model. A model that can't hold the prior decisions while looking at the current one can't actually do the project: it can only do individual tasks and hope the thread holds together between them.
Why multiple sources break a single thread
Real engagements aren't one document. They're a web: the client's own data, market research, analyst reports, competitor filings, industry guidance. The work is in the relationship between sources: does the client's revenue number agree with the market size? does the competitive scan match the analyst reports? And that requires holding several sources in view at once.
A chat model can do this with two or three small documents if you paste them carefully. The moment the sources are large, or numerous, or conflicting, the window pressure returns. The model starts summarising one source to make room for another, and the summary is where the detail dies. Ask it to cross-check a market size against the client's own financials and it will try, but it's trying with compressed versions of both, and cross-checking is exactly the task where compression produces errors.
What this means in practice
The pattern that fails is: one window, one prompt, one giant project. The pattern that works is the one you'd use with a human junior, applied to the AI:
- Break the project into tasks. Each task gets a small, relevant context instead of the whole engagement. This is the single biggest win: it keeps every window small and every answer grounded.
- Give it the document, not the memory. Don't expect the model to remember what you said yesterday. Give it the actual source material for the task at hand.
- Use structure, not conversation. Tables, schemas, structured data: the model handles these better than free-form chat, and they compress less.
- Checkpoint the work. Write the decisions down as you go, in a system the next task can read. The model doesn't persist state; your process has to.
The framing that keeps everyone sane: a chat model is a great specialist and a terrible generalist. Give it one clear task with the right inputs and it can out-perform a junior analyst. Ask it to run the whole engagement in one thread and it will eventually lose the plot: not from malice, but from physics. The context window is a real, physical constraint, and no amount of prompting changes it.
A concrete example
Take building a market-size chapter for a strategy deck. If you paste the whole engagement (all the research, the client data, the analyst reports) into one window and ask for the chapter, the model will produce something plausible, and it will quietly round a figure or two, because it's compressing a mountain of material into a window it can hold. Split the work instead: give it the segment data and ask for the segment sizing; give it the competitor list and ask for the competitive scan; hand it the client's own numbers and ask it to cross-check. Each step stays small, each answer stays grounded, and the final assembly is just stitching together sections the model actually had full access to when it wrote them.
The difference isn't the model's ability. It's the amount of material each answer had to be wrong about.
That's why purpose-built platforms look different from a chat box. They break work into tasks, keep context scoped, store the state outside the window, and hand each model only what it needs for the step in front of it. The model stays fast because it stays small.
Want to see that structure in action? The interactive demo shows a full project decomposed into scoped, context-managed steps, and the scoping tool is the fastest way to feel how a big task becomes small, manageable pieces.
Get new posts and free tools
Join the list: one email when we publish. No spam.