Reading Doesn’t Fill a Database, It Trains Your Internal LLM

Article Comments
21

As a creative writer, I’ve been fascinated by LLM’s since they first came out. I’ve always wondered about how my writing works – when I’m “in the zone,” words and phrases seem to flow out of my brain into my fingers and onto the screen. Occasionally, sure, I stop to actual think about a particular metaphor or word choice, but most of the time the writing just flows. Often I’m shocked and amazed and I wonder how I came up with a particular turn of phrase.

It has struck me long ago that those words and phrases are not really mine – they are a comogulation of everything I have read in my life. Not necessarily the exact phrase, but the style, the pattern, the tone might be from other words, or combined from hundreds of similar things I’ve read.

When LLMs came out I quickly deduced they were working on the same principal. Of course, they have perfect recall and can sometimes regurgitate the exact text they were trained on, which my brain can’t really do. But the concept is similar.

So I’ve never been too upset by the idea that LLMs are “stealing” existing works by being trained upon them – that’s exactly what humans have done for thousands of years. Look at every writer who writes about the authors that influenced them and you can see hints of those previous works in their work. That’s how creativity works. It’s changed, modified, improved, morphed, and combined to make something new, usually without being conscious of the process.

(I’m not convinced AI does anything truly creative, since it doesn’t know what it is doing – there is zero intention – but it is mimicking the human process of creating, for sure.)

This “brain training” is one of the reason I read a lot and try to read different genres of fiction and types of non-fiction. The more I read, the better writer I become.

(If I have a worry about AI, it’s that it is running out of training material. It’s already being trained on its own output, and as more “writing” in the world is AI-generated, the various models will consume that for training. This will water down the content the way photocopies of photocopies are further and further removed from the original. Like a game of telephone, the end result may be corrupted and completely distorted, taking away whatever humanity was in the original.)

1 Like
22

As @Quantumpanda noted, I was just concerning myself with text, but the analogy does extend to other forms of perception, I think.

Well, models are intentional simplifications, so they always fail at some level.

Yes, the chatbots all have some level of “memory” these days, where they remember previous conversations. I’ve found that quite useful. Some of today’s snippet keepers are designed to let you “talk to” what you’ve snipped, and I’m sure that one day, our devices will remember everything we’ve read to help us pull more out. I suppose that will be moving us back toward the database analogy!

Ooo, was that a typo or did you intend to coin “comogulation”?

23

I made it up! :joy:

I don’t know what it is, but I liked it, the meaning seemed clear to me, and I figured I’d leave it as that’s exactly the kind of thing that AI would never do. :wink:

Lots of authors use made-up words* (especially in science fiction). That’s one way we get new words. Maybe it’ll catch on. I wondered if anyone would notice!

* I was just reading a bit by Cory Doctorow about using LLMs for proof-reading his work and he mentioned they don’t like all his made up words. I run into that, too.

2 Likes
24

I approve! Every now and then, a word begs to be coined.

2 Likes
25

My favorite made up word is the transmogrifier from Calvin and Hobbes:

https://calvinandhobbes.fandom.com/wiki/Transmogrifier_Gun

Side note: Years ago I made a text transformer app I use nearly daily which I called Transmogrify:

Article 9107: : Transmogrify Your Text

(The source code is available here, but you need to a Xojo license to compile it into a useable app. I never released it as an app since it’s very geeky, designed for you to write scripts to make your text changes.)

26

I have just read this article and initially took issue with the paragraph beginning “later in life” and then adjusted my opinion. I am 87 with a very cursory education, certainly no university but was an avid reader from a small child. More recently I have significantly changed my opinions on a wide range of issues. In the past this may not have been the case. The reason then came to me “life-changing experience.” My wife died last year. I hope this perhaps adds credence to your article.

4 Likes
27

Conversations like this are an important reason I hang around here, probably more so than just finding out how to fix my latest Apple mistake or learn whether the new latest Apple Whiz-bang is worth my consideration.

2 Likes
28

Bravo @ace for a brilliant article !.
I understand that you want to focus this discussion on the analogy, but the core issue isn’t only the accumulation of (weighted) knowledge —it’s how that knowledge is used.
As the foremost authority on the subject, Hugo Mercier’s central thesis—the Argumentative Theory of Reasoning—reverses the traditional intellectualist view that reason evolved to help individuals think better, make more logical decisions, or find objective truths.
Mercier demonstrates that knowledge is largely used to justify our pre-existing beliefs, to win arguments (“reason as a social tool”), rather than to achieve any kind objective understanding of reality or “truth”, which I find rather depressing.
In his book the Enigma of Reason, Mercier illustrates this concept with the “Lawyer” Metaphor: Reason acts less like a disinterested scientist and more like a lawyer. It seeks “reasons” to defend a client (our own beliefs) and to attack the opposition (others’ beliefs).
~The Argumentative Theory - Edge.org~
~Why do humans reason? Arguments for an argumentative theory~

1 Like
29

i disagree with you! :wink:

2 Likes
30

Luckily we can think unlike the output generated from LLMs. But it might be better to think of our stored knowledge more along the lines of how it is stored in LLMs rather than databases and guess they learned a lot thanks to the study of how we think (both neural networks, but also in concept formation, logic and sentence construction) building the LLMs.

Anyway it is helpful in working with (or writing about) computers to have had some philosophy studies (like me too) – esp. in problem-solving.

31

I’m a cognitive scientist (academically at SFU) and agree that the filing cabinet metaphor is inapt.

I would say however that it’s still important to deliberately instill facts. Productive practice— a concept that combines literature on expertise/deliberate practice, memory retrieval/memory testing effects, and test enhanced learning — can be used to instill facts. When reading, watching or listening, one can look out for knowledge gems in the content, and create “knowledge instillers” for them in powerful flashcard software (like Anki or RemNote), and practice the content at spaced intervals. Research and experience are clear that without deliberate practice there’s a good chance the gems will be forgotten. And it is often valuable to be able to pull out the gems when needed. Also, one sometimes wants to deliberately ensure one can think not only about the information one has “consumed”, but with it. Compare:

Productive Practice: How to Make Information Actually Change You

3 Likes
32

He and his colleague Dan Sperber, also make the point that the mind is somewhat inert, resistant to change. This is why productive practice ( deliberately instilling knowledge) is important. Latter was interviewed by Julia Galef on the argumentative theory in 2015. Great interview:

3 Likes
33

well, it turns out that, as Seth Grant and others have documented, individual neurons are far more complex than those modeled in the current AI systems:

the analogy between LLMs and human mind-braind though interesting is weak.

There’s a flip side to this. A famous cognitive psychologist, John R. Anderson, published an important book The Adaptive Character of Thought (1990) in which he detailed what in my Cognitive Productivity book I called the heuristic relevance-signaling hypothesis. The idea is that evolution faced a major (implicit) challenge: how can one decide what to remember. We can’t simply tell ourselves “I will remember this”. The pre-frontal cortex does not directly control memory in that way. According to our hypothesis, the brain uses as a heuristic signal for deciding what to remember: what one tried to remember in a given day. The calculation happens during sleep, and involves indexing of memory. Simply trying to retrieve information can help you remember it in the future. That’s how productive practice (which I mentioned above) which leverages memory testing effects does its magic.

This is why students who use flashcards tend to do better. Flashcards are not just for rote memory. They serve many purposes. One of them is meta-cognitive: to signal to oneself what one does not know or understand , because one can’t answer the challenge (“question”). In my two Cognitive Productivity I dedicate several chapters to the topic. Flashcard software is gaining in popularity with students, as one would expect, but increasingly knowledge workers are using it for their own learning. I for one practice about 20 minutes a day, instilling all kinds of information. THis is also a way to take control of memory as one ages.

5 Likes
34

Aren’t people fascinating?

And such differences exist, the ability of memory is certainly not evenly distributed, just as the ability to think different is not (ahem). I have one son, a historian, who can retain vast swathes of information, truly an extraordinary capacity to unearth relevant obscure details that were digested months, even years ago, Another, a dreamer and an artist, pulls poetry and wonder out of seemingly nowhere, but can’t recall where he was on Tuesday.

4 Likes
35

I considered majoring in history (rather than psychology) as an undergraduate. One reason I chose psychology is that I didn’t think I could memorize all those dates. Then I realized psychology requires a ton of memorization too. I got by thanks to creating flashcards. That was before test-enhanced learning became a big area of study. I later generalized this into a practice I call productive practice:

The concept combines literature from expertise ( deliberate practice), memory retrieval effects, and test-enhanced learning. The goal is not merely to remember information, but to be able to apply the information when it is relevant. Other goals are to develop skills, habits and attitudes. So I depart far and wide from @ace amorphous memory concept.

The literature on expertise is clear that in many disciplines, particularly public performance disciplines, experts practice retrieving information. I’ve been on a mission to bring this to knowledge work. Through no cause of my own, an increasing number of people use Anki and Remnote more generally – but they are still a super slim minority.

2 Likes
36

Reading trains your internal LLM, so a trained LLM is comparable to a mind educated by reading. But what is an educated brain, or a trained LLM? What relation does educated thought, or a trained LLM, have to reality? What language can, and cannot, tell us was considered by Ludwig Wittgenstein.

The Illusion of Understanding. Wittgenstein, Neuroscience, and Why… | by Stephanie Shen | ILLUMINATION | Apr, 2026 | Medium

LLM computation automates creation of logical-linguistic models of reality, for “reality” as has been described in language. What LLMs do not, and cannot, do is extend linguistic description to as yet undescribed aspects of reality. Such extension requires “intuitive perception” of novel concepts, but what intuition is is not well known. Perhaps examination of the difference between problem solving by LLMs, and problem solving by humans, can better reveal intuition, allowing intuition to be better understood.

A guess is quantum entanglement, and the more applied problem solving by quantum computing, are related to intuition. Conversely, intuition might eventually be understood to be underpinned by quantum entanglement (a mechanistic model of a biological process would help keep reductionists happy). Hypothesizing such a relationship could allow design of experiments testing for the more tractable intuition, which might reveal the more elusive entanglement in their structure. These lines of thought could be extended in many diverse ways, and hopefully will by generations of graduate students creating more knowledge Piled Higher and Deeper.

37

I think the field in question matters a great deal here. In my professional field, the main constant is change—there’s alway something new, so I have to be taking in vast amounts of new information at all times rather than absorbing a relatively static body of information to become expert. I’m constantly developing and refining a gestalt, rather than being able to recall specifics.

I’d argue that what most of think of as being “informed” falls into this category. I don’t need to know huge amounts about even topics that interest me, like EVs, and certainly not about topics that feel important but are outside of my sphere, such as international events.

2 Likes
38

I agree. but some fields are universal, such as the ones I used as running examples throughout Cognitive Productivity: Using Knowledge to Become Profoundly Effective , including how to navigate relationships (I used examples of learning from John Gottman’s famous books: The Seven Principles for Making Marriage Work and The Relationship Cure. Educational psychology is clear that merely reading the books will not as reliably instill the knowledge as practicing. That’s not just for rote learning but instilling the concepts, such as his concept of bid/bid response and harsh startups. (there are more examples in my book, and examples of questions that can be used to master the information). More generally, for everyone: some information is worth mastering. (Divorce rates are high largely because people don’t bother to acquire the skills to make a marriage work.)

1 Like
39

The couple that does flash cards together, stays together. :slight_smile:

John Gottman was at the University of Washington, so we were aware of his work while we lived in Seattle.

1 Like
40

I know it appears counter-intuitive. There’s a common view that test-enhanced learning is rote learning. But the data shows otherwise. E.g., test-enhanced learning provides feedback of what one doesn’t know. It overcomes illusions of learning which are alas the rule in learning. I explain both in detail in chapter 3 of Cognitive Productivity: Using Knowledge to Become Profoundly Effective. There’s also chapters 7, 12-13. sorry to plug the book but it’s not a subject that is easy to summarize. I also wrote: Productive Practice: How to Make Helpful Information Actually Change You but that dooesn’t cover it all. There is also this about the concept of bid and how we respond to them, which is essential in relationships:

From Reading to Changing How We Perceive, Think, Feel and Act – CogZest

Productive practice also makes one articulate knowledge and create mnemonics. Here’s an excerpt from Cognitive Productivity:

Screenshot 2026-04-24 at 14.09.51
Screenshot 2026-04-24 at 14.09.511124×732 50.2 KB

Also from cognitive psychology we know that learning lists is inherently difficult (due to “cue overload” aka “the fan effect”). So we need to break it down. But learning the list is just a step. The key is to think in terms of the content: Have I been clear? have I been polite? etc. Reading one of Gottman’s book will not convert into expertise with the list. One has to rehearse reviewing our experience in relation to the content.

And there are a few other key lists, e.g., in The Relationship Cure: A 5 Step Guide to Strengthening Your Marriage, Family, and Friendships.

Gottman realizes that practice is important, that’s why he gives workshops. But workshops are not always enough for the knowledge to become self-sustaining. The beauty of productive practice is that it keeps knowledge alive. Having said that, it doesn’t take an infinite amount of representation. The spaced learning (a very robust paradigm in cognitive psychology) software is optimized to minimize effort over time.

This is evident from the literature on many domains of expertise: experts don’t just get on stage and perform. They practice. Not all knowledge is worth mastering. But enough is to warrant regular practice in my opinion.

There’s also aging effects: learning becomes more difficult with age, limiting the potential of one-shot learning.