Ask a language model whether any of your contracts expire this year and it will tell you. Ask it which ones and, if you have enough contracts, it can't. Nearly the same words, opposite results. I call the second kind of question an exhaustive query, and the fact that models can't answer them, the Exhaustive Query Problem.
An exhaustive query is a question of the form: give me every X where Y is true. Every contract expiring this year. Every customer who was promised a discount. Every agreement the other side can cancel without cause. People ask these constantly, because they're what you ask when you have to do something rather than know something. You can't renew a contract you didn't find.
The problem has two halves, and each needs to be solved independently. One is finding every X when there are a hundred thousand documents to look through. The other is delivering every X when twenty thousand of them match. It is possible to solve either half without solving the other, and the usual tools solve neither.
Finding
The standard way to connect a model to a pile of documents is retrieval augmented generation, and the standard retrieval is vector search. You turn the question into a vector, turn every chunk of every document into a vector, take the twenty chunks whose vectors are closest, and hand them to the model with the question. When the answer is in those twenty chunks, this works. When it isn't, you have no recourse and no warning. The model answers anyway, fluently, from the wrong or incomplete twenty chunks.
That's acceptable for what I'd call existence queries, where one example answers the question. Has any contract ever had a most-favored-nation clause? If one has, some chunk says so, and that chunk probably sits close to the question. But an exhaustive query is the opposite shape. The twenty it found may not be all of them and vector search can’t tell you which ones it missed. It returns twenty every time. It never says "and two thousand more."
Fig. 1 - Two shapes of question
Vector search returns its twenty nearest chunks every time; enough for existence, silent about what it missed.
It fails for a second reason too. "Expiring this year" isn't a phrase in any contract. What's in the contract is a term clause with a date and a duration, and deciding whether those add up to this year is arithmetic, not semantic similarity. Vector search finds text that sounds like your question. Exhaustive queries are usually about a property of the document, a date or an amount or a party, and a property doesn't sound like anything.
A newer approach is to give the model a loop and let it search the files itself, the way a person would. This is better, because it can keep going. But a person asked to check a hundred thousand contracts wouldn't read them. They'd ask for a spreadsheet. An agent that does read them takes hours, and the list it builds along the way is itself twenty thousand items long, which is more than it can hold in its head. So it runs out of time, or it runs out of memory, and either way it stops before the end without quite knowing it stopped.
Fig. 2 - The agent loop
An agent that reads the corpus fills its context before it reaches the end.
It's also expensive in a way that's easy to miss. An agent is billed by the token, and it reads a contract by pushing the whole contract through the model, so the cost of a question is the cost of reading the entire corpus, whether the answer turns out to be three contracts or twenty thousand. More than that, actually, because each step of the loop carries along everything the agent has seen so far, so the thousandth contract costs more to read than the first. You can make it forget as it goes to hold the bill down, but then it's forgetting, which is the failure you were trying to avoid. And none of the work is kept. Ask a slightly different question tomorrow and the agent starts from zero and pays for the whole corpus again.
Delivering
Suppose you did find all twenty thousand. A model can emit only so many tokens in one answer, and twenty thousand contract names is more than that. It isn't inconvenient. It's impossible. For a sense of scale, about 25,000 asteroids have names and more than 700,000 have numbers, and no model can recite either list. [1] The companies we build for have thousands to hundreds of thousands of contracts, so their answers regularly land in that range. And even if a model could produce a twenty-thousand-line chat message, you wouldn't want one. A list that long isn't something you read. It's something you sort, filter, and export, and a chat bubble does none of those.
Fig. 3 - The output limit
Twenty thousand rows don't fit in a chat bubble - and a chat bubble can't sort, filter, or export them anyway.
The fix
So both approaches ask the model to do the two things it's worst at: hold everything and say everything. What it's good at is reading one document carefully and understanding what you meant. The fix follows from that, and it's what we built in Contract Intelligence.
To solve the finding problem, I used a standard, highly reliable tool: a database. A database is designed to be complete and thorough. If you ask it for every contract expiring in 2026, it will give you every single one without missing any. However, it can only search through information it already stores. To handle this, our AI reads each contract as soon as it's uploaded and extracts key details–such as parties involved, key dates, and renewal terms–into organized fields. The AI handles the reading, while the database manages the storage and searching, playing to each tool's strengths.
Of course, you can't predict every piece of information you might need later. Eventually, someone asks for a detail the system didn't automatically save. To fix this, you can just ask the system to track that new detail, and it will go back to scan every contract to pull it out. That initial scan takes a bit of work, but it only happens once. From then on, looking up that information is instant and effortless.
For output, we built a web interface the model can control. Instead of typing rows into a chat, the model turns the question into a query, decides how the results should be shown, and lets the rows flow from the database to the screen. They never pass through the context window, so its size stops mattering. Twenty results or twenty thousand, the model's share of the work is the same.
Fig. 4 - Model at the edges, database in the center
The model reads on the way in and decides what to show on the way out. Rows never pass through it.
