Context, Cost and Limits — What AI Research Actually Costs You

A context window is the amount of text a model can hold in view at one time. Think of it as a desk, not a filing cabinet. A filing cabinet stores things you can fetch later. A desk holds only what fits on the surface right now, and when you put a new document down, something else gets pushed off the edge.

The important part is what shares that desk. It is not just the document you pasted. It is also the instructions the provider wrapped around your session, every question you have asked in this conversation, every answer the model has already given, and the reply it is currently composing. Four separate things compete for the same fixed space, and only one of them is the thing you actually care about.

This is why the same prompt can work beautifully in a fresh chat and disappoint in an old one. Nothing about the prompt changed. What changed is how much of the desk was already occupied before your question landed on it. In a thread that has been running for forty exchanges, your carefully written prompt is arriving into a crowded room.

Exact window sizes differ by provider and change every few months, so there is no useful number to memorise here. What does not change is the shape of the problem: the space is finite, it is shared, and it refills every turn. Design your sessions around that and the specific limit stops mattering.

There are two different failures here and it is worth keeping them apart, because only one of them is visible. The first is the hard limit. You paste more than the window holds, and the interface either refuses the message or silently keeps part of it. The second is the soft failure. Everything technically fits, the model answers cheerfully, and the answer is worse than it would have been from a tenth of the material.

The silent version is the dangerous one. When a long paste gets truncated, the model does not say "I only received the first half." It answers using what it has. So you ask about the related-party transactions disclosed in the notes near the back, and you get a fluent paragraph built out of the sections that survived — or out of the model’s general knowledge of what such a note usually says. Both read identically.

Even where nothing is truncated, size costs you accuracy. An Indian annual report is a mixed document: a chairman’s letter, a management discussion section, the standalone and consolidated financial statements, the notes, the auditor’s report, a corporate-governance section, and a large amount of statutory boilerplate that is nearly identical year to year. Ask one question across all of that and you are asking the model to find a specific sentence in a haystack it was handed all at once.

The fix is not a bigger window. The fix is deciding, before you paste anything, which part of the document could actually contain your answer. Segment revenue lives in the notes to the accounts. Management’s stated risks live in the management discussion. An emphasis-of-matter remark lives in the auditor’s report. Paste that section. Leave the rest on the shelf.

Even inside a paste that fits comfortably, the model does not treat every line as equally important. Material at the very beginning and the very end of a long input tends to be used more reliably than material buried deep in the middle. Researchers call this the "lost in the middle" effect, and it shows up across model families rather than being a quirk of one product.

The everyday version of this is familiar. Hand someone a twenty-page document and ask them what it said, and they will reliably remember the opening and the conclusion. The middle blurs. The model’s version is not identical to human memory, but the practical shape is close enough that the analogy earns its place.

This has two direct consequences for how you write a research prompt. Put your instruction at the top, before the pasted source, so it frames everything that follows. Then repeat the specific question at the very bottom, after the source, so it is the last thing in view when the model starts writing. The source sits between two copies of the same request.

The second consequence is more important. If your pasted block is short enough that there is no deep middle, the effect largely stops mattering. This is the real argument for splitting a document — not that the pieces fit, but that in a small piece every line is near an edge. A four-page extract has no middle to get lost in.

Each time you send a message, the model re-reads the entire conversation from the beginning. Your first question, its first answer, your correction, its apology, and everything since. That history is not a summary. It is the raw transcript, and it occupies the same window as the document you are about to paste.

The mechanical cost is obvious: less room for new material with every turn. The subtler cost is contamination. If the model stated a wrong figure in turn six and you did not catch it, that wrong figure is still sitting in the transcript at turn thirty. From the model’s point of view it is now established context — something already agreed in this conversation — and it will build on it rather than question it.

Instruction drift compounds the same way. You asked for a table early on, then asked for prose, then asked it to be more concise, then asked for more detail. All four instructions are still in view and partially contradictory. The model resolves that conflict silently, usually by favouring whatever you said most recently, and you lose the formatting discipline you set up at the start.

The remedy is unglamorous. Start a new thread far more often than feels necessary. A thread should cover one document or one narrow question, and when the subject changes, the thread should change with it. Threads are free. Reusing one because it has "context" is how bad context gets preserved.

Splitting sounds like a chore until you do it twice. It is mostly mechanical, it takes a few minutes, and it converts an unreliable single answer into a set of small answers you can actually check. The goal is one self-contained extract per prompt, each carrying enough surrounding text that a figure is not stripped of its label.

Split along the document’s own structure, never by page count. An annual report already tells you where its joints are: management discussion, standalone financials, consolidated financials, notes to the accounts, auditor’s report, related-party disclosures, corporate governance. Cutting mid-table to hit an arbitrary length is how a number gets separated from the row heading that gives it meaning.

Overlap the cuts slightly. If a note begins on one page and its table continues onto the next, include both in the same extract. A number without its unit, its period label and its comparative column is not a fact — it is a digit. This is the single most common way a split introduces an error that the original document never had.

Then run the same extraction prompt against each piece and keep the outputs in one file of your own, not in the chat. The chat is scratch space. Your notes file is the deliverable, and it is the thing you will still be able to read in six months.

Starting a new thread is only cheap if you can restart quickly. That is what a handoff note is for: a short block of verified material you paste at the top of the new thread so you are not rebuilding from zero. It is the only thing that should cross from one thread to the next.

A handoff note carries facts you have checked, each with its source and page reference. It carries the output format you are using, so the new thread produces something that matches the old one. It carries the open questions you have not answered yet. That is all. Three items, usually under twenty lines.

What must not cross is the model’s unverified prose. If it wrote three paragraphs of analysis in the old thread and you never checked the numbers underneath, carrying that forward carries the errors forward with it and launders them in the process — in the new thread they arrive as premises rather than as claims, and premises do not get questioned.

The discipline that makes this work is doing the verification before the handoff, not after. A fact goes into the note once you have seen it in the source with your own eyes. Everything else stays in the old thread and dies there, which is exactly what you want.

For a retail trader, the money is the least interesting part of this. A consumer subscription is a fixed monthly number you can look up before you commit, and it does not vary with how hard you use it. Pay-per-use API access varies with volume, but the volume one person generates reading filings is small. The bill is rarely the constraint.

The real cost is your attention, and it splits in two. There is the time you spend setting up — splitting documents, writing the prompt, pasting extracts. That cost falls sharply once you have a saved template, because you stop composing and start filling in blanks. Then there is the time you spend verifying, and that cost does not fall. It is proportional to how many claims you intend to rely on, and it stays proportional forever.

This is the trade the tool actually offers you. It compresses reading time significantly and it adds verification time. If you take the compression and skip the verification, you have not saved anything — you have swapped a slow honest process for a fast unreliable one, and the switch is invisible until a number you never checked turns up in a decision.

There is a third cost that never appears on any bill: the cost of a wrong claim you did not catch. It is rare, it is unpredictable, and it is the only one of the three that can be genuinely expensive. Every habit in this module exists to keep that number at zero.

Everything above collapses into a small set of habits. One document per thread. One section per prompt. A fixed prompt you do not rewrite each time. Verified output copied out of the chat and into a file you own. A handoff note when you stop.

The reason to make these habits rather than decisions is that all the constraints in this article fail silently. There is no alert when the window gets crowded, no warning when a fact slips into the unattended middle, no notification that turn thirty is less reliable than turn three. A process that only holds when you remember to apply it will not hold on the afternoon you are tired and in a hurry.

It is also worth being honest about the sessions where this is overkill. Asking a model to explain what an emphasis-of-matter paragraph is does not need a split document or a handoff note. The structure earns its keep when the output is going to inform a decision. Match the ceremony to the stakes.

One final point about time. Working this way is slower per document than pasting everything and hoping. It is considerably faster than reading 300 pages yourself, and unlike the hopeful version, what you end up with is checkable. That is the whole proposition, and it only holds if the checking actually happens.