A $200 GPU Beat GPT-4o at Reading Spreadsheets

the actual problem nobody talks about

Real company data is NOT sitting in some perfectly indexed knowledge base. It's in a 47-tab Excel workbook that Nadine from Finance made in 2019. It's in CSV exports from a legacy ERP system that nobody fully understands. It's in quarterly PDFs where the numbers on page 12 only make sense if you cross-reference the footnotes on page 34. (ask me how I know)

And every AI vendor's pitch is the same: "just throw it in RAG." Chunk your docs, embed them, vector database, done. The AI will figure it out. Sure it will.

Here's what ACTUALLY happens. RAG takes your beautifully structured spreadsheet and shreds it into 500-token fragments. A movie's budget ends up in one chunk. The column headers end up somewhere else. The title that tells you which film even had that budget? Gone. Then the retrieval engine picks whichever chunk has the most words in common with your question and calls it a day. That's it. That's the "intelligence."

chunking destroys the structure RAG needs to answer correctly

ok but RAG works tho?

For text, yeah. 200-page policy doc and you want to find the parental leave section? RAG nails that. Blog posts, docs, FAQ databases, all solid use cases. No complaints.

The problem is everyone saw RAG work for text and assumed it works for EVERYTHING. Structured data. Multi-sheet workbooks. Financial reports where table A references table B. RAG doesn't just underperform on these, it actively destroys the relationships that make the data meaningful. That's not a minor issue, that's the whole point of the data.

It can't compare values across rows. It can't compare budgets across movies when their data lives in different chunks. And every time your data changes? Re-embed the whole thing. Fun.

RLM: the model just reads your data

Recursive Language Models flip the whole thing. Instead of pre-chewing your data into chunks and praying the right piece floats to the top, RLM lets the model do what YOU'D do: open the file, look at the columns, filter the rows that match, check another table if needed, come back with an answer. Wild concept, I know.

The model writes code on the fly, like df[df.budget > 100], and executes it. Basically applying a filter in Excel, except the model writes the filter itself. Answer not in the first table? It checks the next one. Needs to compare values across rows or aggregate them? It writes the logic. It's recursive: the model calls itself on subsets of the data until it has what it needs.

RAG is Ctrl+F on a shredded document. RLM is handing the original to someone who can actually read.

the model navigates, not searches

Paper: Recursive Language Models (arxiv 2512.24601)

why you should care

There's a practical angle here that nobody talks about.

RAG stuffs your context window with retrieved chunks, most of which are irrelevant. You're paying for tokens the model has to read through just to find the one row it actually needs. With a cloud API, that's real money. With a local model, that's wasted compute and slower responses. Either way you're burning resources on garbage.

RLM reads surgically. Load the schema, write a targeted query, only process the data that answers your question. On structured datasets this means DRAMATICALLY less context for the same (or better) answer quality.

Here's the fun part: I ran a 9B parameter Qwen model with RLM on a consumer GPU, an RTX 3060 you can buy used for $200, and it consistently beat GPT-4o using RAG on structured data tasks. Not because the model is smarter. Because it's reading smarter. A $200 GPU beating a $20/month API. That's not hype, I tested it.

ok enough talking, try it

Enough words. Below is a live comparison: same question, same data, three approaches running in real time. Vanilla LLM, RAG, and RLM. Switch datasets, switch models, see for yourself which one actually gets it right. Don't take my word for it.

Side note: try the AI models dataset and ask who has the most models. The vanilla LLM with zero context might think you're asking about fashion models. The RAG one pulls a couple chunks but still struggles to count across rows. The RLM one actually runs commands to filter and count the data, like a human applying Excel filters. That's the whole point.