Originally published at: Reading Doesn’t Fill a Database, It Trains Your Internal LLM - TidBITS
I had an interesting conversation with my son Tristan the other day. Because he’s so engrossed in his PhD research in machine learning at Simon Fraser University, I often try to steer our discussions away from the nitty-gritty of his experiments and toward more general tech topics I can grasp without graduate-level math and computer science. We were chatting while I was eating lunch, usually a time when I read a magazine or newspaper, and something he said made me wonder out loud, “Why do I read?”
That’s an existential question, since I read constantly throughout the day. For some types of reading, the answer is easy. I read the local alt-weekly newspapers because of their real-world connections to the people, institutions, and environment in which I live. Before bed, I read fiction for enjoyment and to shift my mind away from the thoughts of the day to help me get to sleep. And I keep up with tech news because it’s my profession—I need to know what’s going on even when it likely won’t affect what I write in TidBITS directly.
Harder to explain are The New Yorker, Science News, and other magazines my mother enjoys giving to me after she’s done, evergreen articles in old copies of The New York Times that a friend of my parents saves for me to start twice-daily fires in our kachelofen woodstove, and RSS-retrieved blog posts on a variety of topics (see “Comparing Blogtrottr, Feedrabbit, and Follow.it for Receiving RSS Feeds in Email,” 22 August 2024).
Perhaps I’m Building a Database?
In the past, I’ve thought of reading as a form of database import. The more information I consumed and added to my internal database, the more I would know, the better my writing would become, and the more scintillating a conversationalist I’d be. And somehow fame and fortune would follow. I’m apparently not very good at long-term goals.
But “know” is a loaded word—even though I have an objectively decent memory (for facts, if not events and emotions, perhaps related to my aphantasia), I’m sure that I forget nearly everything I read. Just because I’ve read an article doesn’t mean I could tell you much about it a week, a month, or a year later. Arguably, if I went back to an article I read a year ago, I might not even remember having read it before. Heck, I don’t even necessarily remember what I’ve written a few years later—that’s what the TidBITS search engine is for.
I’m well aware of the ephemeral nature of memory, so I periodically investigate apps or services that let me save bits of text that seem particularly insightful or important while I’m reading and that I’m certain that I’ll want to refer back to at some future point in time. The latest one is Sublime, but, as with all its predecessors, I started clipping text to it, got busy with something else, and stopped using it before I ever came up with a reason to search through my snippets. I never go back to these apps or the information I thought was so important in the past.
The one database I do maintain is my email archive. I save nearly all my email in Gmail, and I regularly search for old conversations, largely by person, to revisit the topics. However, I seldom use it to return to articles, blog posts, and newsletters. I have to mark messages containing general information unread if I want to refer to them in the near future. Once a message has been marked as read, I’m unlikely ever to remember it or see it again, no matter how important I initially thought it was. Ironically, I’ve accumulated so many unread messages that I’ve forgotten why most of them seemed worth saving.
Or Maybe Training a Large Language Model?
The realization from my conversation with Tristan is that what reading really does is adjust the weights in my internal large language model. Let me explain.
Briefly, large language models are trained by feeding them enormous amounts of text and asking them to predict what word comes next in a known sequence. When the model’s guess doesn’t match the actual training data, its internal “weights”—the billions of numeric values that map the links between concepts—are nudged slightly to make the correct answer more likely next time. After billions of these adjustments, the weights encode useful patterns: how ideas relate to each other, what concepts cluster together, what kinds of responses make sense in different contexts.
There’s a recursive irony here. We have long tried to understand—or at least talk about—the brain by comparing it to prominent technologies of the era: telephone switchboards, filing cabinets, databases. Now I’m comparing my mind to an LLM, but neural networks were themselves loosely inspired by how we think biological neurons work. The metaphor loops back on itself, which perhaps suggests it’s less of a metaphor than it first appears.
As a dedicated reader, I’ve consumed vast quantities of text—perhaps several thousand books, more than a hundred thousand articles, and over a million email messages, though I shudder to do the math. While my consumption of text pales in comparison to even a toy LLM, the analogy feels more apt than a database. I’m not adding records to a mental database; I’m subtly adjusting the likelihood that certain ideas, phrasings, and connections will surface when I think, speak, or write.
Reading a debunking of data centers in space doesn’t mean I’ll remember (or even understand) the equations behind why the idea is flawed, but it will probably update my training data from high school physics to nudge me more in the direction of skepticism the next time someone proposes solving an Earth-bound problem by launching it into orbit. Reading widely—even material I’ll mostly forget—keeps reweighting my internal model, shaping what I reach for without my conscious awareness.
Extending the LLM Analogy
This analogy even maps pretty well to how we learn. As children, we essentially pre-train our models on general data and build foundational weights—our connections between core concepts. Since they’re based on relatively little training data, those weights have less substance and are more easily affected by new information. Reading a single book or taking an influential class can radically change our views on the world.
During formal education and professional training, reading to master a subject works more like fine-tuning a large language model. With fine-tuning, the model is further trained on a smaller, specialized dataset. People learning new fields benefit from repetition, active recall, and deliberate engagement precisely because they’re trying to create strong new weights where few existed before.
Later in life, most of those weights are sufficiently mature that the flow of general reading can adjust them only slightly. An older person is likely to adopt a previously unthinkable position only if they have a life-changing experience or go down the rabbit hole for a particular topic.
Flipping my internal analogy from database to large language model is surprisingly freeing. No longer do I have to decide whether something I’m reading is important enough to bookmark, file away in a snippet keeper, or mark in my email app. Beyond the desire to keep items near the surface when I want to write about them in the near future, I can let what I read go in one eye and out the other, adjusting my mental model’s weights along the way.
To draw on my background as a Classics major at Cornell (a few strong weights from my Ancient Philosophy classes!), the analogy is almost Heraclitean in its elegance. Heraclitus is often paraphrased as saying, “No man can step in the same river twice,” calling attention to the fact that neither the river nor the man (at least in later interpretations) remains the same on any subsequent immersion. Information is a stream through my consciousness, and every particular bit reshapes my consciousness ever so slightly in passing by.
Will I be able to pull out an accurate retelling of what I’ve read at what’s called “inference”—when a model generates output in response to a prompt? Maybe, maybe not. Human memory is fallible in much the same ways that AIs hallucinate, though we usually call our hallucinations “anecdotes.” But if something tickles enough of my neurons that I can trigger a search to inform what I’m writing or make a devastatingly apropos comment in a cocktail party conversation, I’m happy.
If you, like me, have ever felt guilty about remembering little of what you read, perhaps it’s worth reframing: you’re not failing to build a database—you’re tuning your personal LLM.
