Reading Doesn’t Fill a Database, It Trains Your Internal LLM

Article Comments
21

As a creative writer, I’ve been fascinated by LLM’s since they first came out. I’ve always wondered about how my writing works – when I’m “in the zone,” words and phrases seem to flow out of my brain into my fingers and onto the screen. Occasionally, sure, I stop to actual think about a particular metaphor or word choice, but most of the time the writing just flows. Often I’m shocked and amazed and I wonder how I came up with a particular turn of phrase.

It has struck me long ago that those words and phrases are not really mine – they are a comogulation of everything I have read in my life. Not necessarily the exact phrase, but the style, the pattern, the tone might be from other words, or combined from hundreds of similar things I’ve read.

When LLMs came out I quickly deduced they were working on the same principal. Of course, they have perfect recall and can sometimes regurgitate the exact text they were trained on, which my brain can’t really do. But the concept is similar.

So I’ve never been too upset by the idea that LLMs are “stealing” existing works by being trained upon them – that’s exactly what humans have done for thousands of years. Look at every writer who writes about the authors that influenced them and you can see hints of those previous works in their work. That’s how creativity works. It’s changed, modified, improved, morphed, and combined to make something new, usually without being conscious of the process.

(I’m not convinced AI does anything truly creative, since it doesn’t know what it is doing – there is zero intention – but it is mimicking the human process of creating, for sure.)

This “brain training” is one of the reason I read a lot and try to read different genres of fiction and types of non-fiction. The more I read, the better writer I become.

(If I have a worry about AI, it’s that it is running out of training material. It’s already being trained on its own output, and as more “writing” in the world is AI-generated, the various models will consume that for training. This will water down the content the way photocopies of photocopies are further and further removed from the original. Like a game of telephone, the end result may be corrupted and completely distorted, taking away whatever humanity was in the original.)

1 Like
22

As @Quantumpanda noted, I was just concerning myself with text, but the analogy does extend to other forms of perception, I think.

Well, models are intentional simplifications, so they always fail at some level.

Yes, the chatbots all have some level of “memory” these days, where they remember previous conversations. I’ve found that quite useful. Some of today’s snippet keepers are designed to let you “talk to” what you’ve snipped, and I’m sure that one day, our devices will remember everything we’ve read to help us pull more out. I suppose that will be moving us back toward the database analogy!

Ooo, was that a typo or did you intend to coin “comogulation”?

23

I made it up! :joy:

I don’t know what it is, but I liked it, the meaning seemed clear to me, and I figured I’d leave it as that’s exactly the kind of thing that AI would never do. :wink:

Lots of authors use made-up words* (especially in science fiction). That’s one way we get new words. Maybe it’ll catch on. I wondered if anyone would notice!

* I was just reading a bit by Cory Doctorow about using LLMs for proof-reading his work and he mentioned they don’t like all his made up words. I run into that, too.

2 Likes
24

I approve! Every now and then, a word begs to be coined.

2 Likes
25

My favorite made up word is the transmogrifier from Calvin and Hobbes:

https://calvinandhobbes.fandom.com/wiki/Transmogrifier_Gun

Side note: Years ago I made a text transformer app I use nearly daily which I called Transmogrify:

Article 9107: : Transmogrify Your Text

(The source code is available here, but you need to a Xojo license to compile it into a useable app. I never released it as an app since it’s very geeky, designed for you to write scripts to make your text changes.)

26

I have just read this article and initially took issue with the paragraph beginning “later in life” and then adjusted my opinion. I am 87 with a very cursory education, certainly no university but was an avid reader from a small child. More recently I have significantly changed my opinions on a wide range of issues. In the past this may not have been the case. The reason then came to me “life-changing experience.” My wife died last year. I hope this perhaps adds credence to your article.

4 Likes
27

Conversations like this are an important reason I hang around here, probably more so than just finding out how to fix my latest Apple mistake or learn whether the new latest Apple Whiz-bang is worth my consideration.

1 Like
28

Bravo @ace for a brilliant article !.
I understand that you want to focus this discussion on the analogy, but the core issue isn’t only the accumulation of (weighted) knowledge —it’s how that knowledge is used.
As the foremost authority on the subject, Hugo Mercier’s central thesis—the Argumentative Theory of Reasoning—reverses the traditional intellectualist view that reason evolved to help individuals think better, make more logical decisions, or find objective truths.
Mercier demonstrates that knowledge is largely used to justify our pre-existing beliefs, to win arguments (“reason as a social tool”), rather than to achieve any kind objective understanding of reality or “truth”, which I find rather depressing.
In his book the Enigma of Reason, Mercier illustrates this concept with the “Lawyer” Metaphor: Reason acts less like a disinterested scientist and more like a lawyer. It seeks “reasons” to defend a client (our own beliefs) and to attack the opposition (others’ beliefs).
~The Argumentative Theory - Edge.org~
~Why do humans reason? Arguments for an argumentative theory~

1 Like
29

i disagree with you! :wink:

1 Like
30

Luckily we can think unlike the output generated from LLMs. But it might be better to think of our stored knowledge more along the lines of how it is stored in LLMs rather than databases and guess they learned a lot thanks to the study of how we think (both neural networks, but also in concept formation, logic and sentence construction) building the LLMs.

Anyway it is helpful in working with (or writing about) computers to have had some philosophy studies (like me too) – esp. in problem-solving.