CompAG: Compile-Augmented Generation
Retrieval-Augmented Generation treats memory as a search problem: embed the corpus, embed the query, retrieve the top matches, and hand them to a language model to read. The model then reads, counts, date-orders, reconciles, and deduplicates that text on the fly — for every query, forever.
It works, until it doesn't. And when it fails, it fails in the same few places every team eventually hits.
The RAG ceiling
Arithmetic. “How many times did I mention X?” asks the model to count occurrences across retrieved chunks while deduplicating near-identical ones. Models miscount — and retrieval truncation can make the count wrong before the model even sees the data.
Time. “When did X happen relative to Y?” needs date normalization (“last Tuesday”, “two months ago”) and interval math. LLMs do date arithmetic in token space, and they are demonstrably poor at it.
Currentness. “What is my current setup?” requires knowing which of several conflicting statements is still live. Retrieval has no notion of supersession; it confuses recency-of-writing with recency-of-fact.
Provenance. The best RAG can cite is a document. It cannot prove that a specific claim came from specific bytes, unmodified.
The usual responses — bigger context windows, rerankers, graph-RAG, agentic multi-hop retrieval — all optimize the query-time reading. They make the model read better, or read less, or read twice. None of them remove the property that the reasoning itself is performed by a stochastic reader, at query time, on every request. The ceiling is architectural. It cannot be prompted away.
The move: understand once
CompAG — Compile-Augmented Generation — inverts the bet. Instead of understanding at query time, it does the hard understanding once, at compile time, and serves queries from verified finished goods rather than raw material.
The rule that makes it work: a query is answered from compiled artifacts — facts, entities, calendars, aggregates — not from re-read source text. If an answer is computable (a count, a sum, an ordering), it is computed in code, not guessed by the model. And every served fact must trace, byte-for-byte, back to its source.
The flow
Four steps, and only the first one costs real work — and it's paid once per document, not once per query.
Why it changes the outcome
Because the reading happens once, queries are fast. Because you send conclusions instead of raw context, they are cheap. Because every fact carries its exact source range, answers are provable. And because facts are versioned in time, the answer is current — the old fact is superseded, dated, not overwritten.
“CompAG doesn't make the model better at counting. It makes the model's counting ability irrelevant.”
That is the whole thesis in one sentence. On the computed path, a wrong count isn't unlikely — it's structurally impossible.
On honesty
A claim like this deserves a number, and numbers deserve a fair test. We're publishing the evaluation methodology before the results — including a no-leak protocol that forces retrieval to actually find the evidence rather than having it planted in context. A design frozen before measurement is the only kind we're comfortable putting a number next to. When the run is done, it's in the paper.
Until then, the argument stands on its own terms: the move from query-time understanding to compile-time understanding is architectural, not a prompt trick. That's the part that doesn't need a benchmark to be true.