I disagree. Finding its own solution to a stated problem is what ALL machine learning does.
When you train a model with a million photos, tagged with the contents, and it configures its own internal weights based on feedback from repeated iterations through the training, that’s the exact same thing.
Nobody is manually programming all the weights in the model, and (aside from the most trivial models), nobody can even understand the logic encoded in those weights. But it works (usually, if the model is designed and trained well).
I understand what you’re saying, but large language models (from what I’ve read) simply deal with probabilities. They don’t actually learn anything. Geoffrey Hinton’s robots learn; so I don’t understand your assertion that all machine learning is doing the same thing.
I would argue that they are more similar than you realize.
The nature of neural nets (I assume everybody is using NNs, since other AI/ML tech is not very popular these days) is that they are “trained” on basic data. The training for a traditional model takes a form something like:
Start with a large collection of annotated data. That is, input data (images, sounds, whatever), that is paired with the correct answer (e.g. identifying what’s in the image, or what word was said in the audio, etc.). There will typically be hundreds or thousands (or more) objects for this.
You run most of the data (the “training data”) through the model and apply an “error function” that estimates how good or bad the model’s results were.
You then use the results to push modifications to the model’s weights (that is, the factors each neuron applies to its inputs in order to generate its outputs) based on the expected result and the amount of error. These typically make very small changes so the results of one object doesn’t obliterate the results from other objects.
Repeat the previous two steps a lot. Hundreds or thousands of times. If your training scripts and model are well-designed, the overall error should decrease with each iteration.
Periodically run the model against the data you didn’t use for training in order to evaluate it against previously-unseen content. Do not use the results of this data for modifying the neurons. (This is to make sure it doesn’t get hyper-specialized such that it can only correctly process the specific objects in the training data.)
Eventually, when the error gets small enough, you release the model (with the trained weights) for use against real data, at which time, the weights no longer change.
All neural nets are trained in a fashion similar to this. And all but the most trivial provide as outputs a huge list of possible choices (list of words that could be recognized, rectangles around where objects might be found, etc.) along with a “confidence” for each one. The software using the model will typically filter out all results whose confidence is less than a configured threshold and return the remaining results (or maybe just the one with the most confidence) as output to the user.
Image recognition models do this - presenting a big array of confidence values for every object is is trained to recognize. Object recognition models also do - presenting an array of “bounding boxes” around regions of the image, along with an object type and confidence. So do LLMs - presenting many words or phrases likely to come next, along with confidences. And so does the hockey-robot - outputting various decisions (probably motor-control commands), with those probabilities.
In all cases, the ML model stops there. It then falls to the software using the model to select the best action, using criteria not built-in to the model in order to generate output (e.g. an annotated image, or lines of text, or servo motor positioning parameters).
Models that train themselves (e.g. the hockey robot, but also many other models designed for self-learning) don’t go through the exact same system of training - they generate their error factors based on the results of their action (where did the puck go, did the robot insect move forward or backward or fall over, etc.), but ultimately it is still a training/feedback loop that gradually modifies neuron weights over time until the developer decides it is good enough and locks down those weights.
Easily said, but pure science fiction. For this to happen, you would have to postulate a future where complete control over genocide-level technologies had been handed over to an AI that has the safety guidelines of a hockey robot.
We’re talking about writing tools here, where AI will help with things like auto-correcting the desired homophone, ensuring generally agreed-upon grammar, and offering alternative wordings that vary word choice. And yes, it can generate fluent text based on a prompt, though it’s often quite obvious and not terribly good. That’s light years from predicting the end of the human race.
There’s no question that AI is being used poorly, but that’s the fault of the scientists who are submitting papers that they clearly didn’t write. I mean, if you can’t even be bothered to remove the telltale “as of my last knowledge update” phrase from your paper…
But even the title of the Scientific American article gives the wrong impression. AI chatbots haven’t “infiltrated” anything because they can’t do anything on their own. They were used badly by people who should have known better.
Also easily said. AI developers are hungry for profit, and perhaps for glory—not regulation, even if they claim otherwise. If they really cared about regulating this powerful technology, strong guardrails would already be in place. Current AI technology is already in the hands of bad actors because of lax security protocols. I think the truth is that nobody knows where this is going next.
I agree with that. I think especially people with a tech background like to focus on potential for good and easily veer off into being somewhere between uncritical and complacent. It’s this illusion of being able to “change the world” that seems to offer great allure.
Back when social media came up a lot of people warned about the risks of giving any uneducated ignorant fool a megaphone. But every tech bro from Santa Clara to Helsinki seemed convinced that this was going to be a great boon for free speech and democracy and empower the masses and bla bla. Those who dared point at the downsides of this development, the addiction to likes, the bubbles, and the change this brought to discourse in general were ridiculed as luddites. And then Jan 6 came around. Is that close enough to a coup in the world’s oldest democracy for people to finally get it? No, the same cheery crowd is now arguing Jan 6 and Trump and all that other nonsense would have happened just then same even without Facebook and Twitter and all that other garbage. Yeah, right. So here we go again. This time it’s AI. So time for the next iteration of some Michael Moore 5th grader tech libertarianism. You can’t stop tech. It’s all for the good of humanity. It just needs to be used right by the right people. That same story has been told probably for as long as humans have developed. It’s as naive as it always was.
While obviously I haven’t tried Apple’s writing tools yet (they aren’t available), I am curious and a little bit skeptical. I have experimented with ChatGPT and the biggest problem I have found is inconsistency.
Ask ChatGPT to proofread the same text six times and you’ll get six different responses. (Some of the same errors will be marked, but some obvious ones won’t be, and each will have a correction or two that aren’t in the others.)
It’s possible I don’t know what I’m doing well enough with my prompts and customizing ChatGPT to do what I ask and a trained and specific system like Apple’s should work much better, but I still wonder how reliable this will be if it can’t answer the same question the same way every time.
(I think LLM’s are programmed that way, so as it not sound repetitive, but if something is fact – like a word misspelled – the AI shouldn’t be creative and sometimes report it and sometimes not.)
It’s not an easy read, but it walks you through the basics of how neural networks work, where the only prerequisite knowledge is the basics about how to write code in Python.
It starts from first principles and has you develop an increasingly-complicated library of code in order to solve a basic (but not trivial) problem (identifying three nested spirals from a pile of generated points).
You won’t be recognizing speech or scanning photos for your friends, but it will get you familiar with the basics on which more advanced neural net software packages (e.g. TensorFlow, PyTorch and ONNX) are all built. Not enough to immediately start inventing cool stuff, but enough so you won’t be completely lost, should you decide to start reading papers from the people who are inventing cool stuff.
The big AI companies have been calling for regulation because they know that large companies can much more easily meet regulatory requirements than small startups. It’s a long-standing competitive strategy. Plus, it gives them a seat at the table in shaping the regulations and makes them seem more trustworthy to the public, where they suffer now. It feels a lot like Br’er Rabbit and the briar patch.
No, it’s widely understood technology that’s taught at university. All bad actors have to do is hire people with machine learning experience. No one is suggesting export restrictions on AI technology.
That’s a job for Grammarly, which does it the same every time. I know, because I check every article multiple times in Google Docs and sometimes in WordPress as well, and I have to ignore the things I disagree with each time. Grammarly is a lot more like what Apple Intelligence will do than ChatGPT.
You were saying that the the bad actors had the technology “because of lax security protocols,” which implies that if there had not been lax security protocols, the bad actors wouldn’t have gotten it. I was pointing out that security isn’t a part of the question at all—this is common technical knowledge. In other words, the genie is out of the bottle.
But let’s bring this back to the discussion of AI writing tools, which bad actors will merely use for their term papers because they’re spending too much time in drama class.
Well, the genie is definitely out of the bottle; but I do not believe that “common technical knowledge” built the AI systems that some of us are referring to. I agree with you, however, that this discussion has ranged far beyond the original topic; so by all means let us focus on Apple’s AI-assisted writing tools.
No, I’m sure it didn’t, but when guns and Germans get mentioned in the same post, even in a contrary example, the early warning systems for Godwin’s Law start to go off for me.
But seriously, if people want to discuss things here other than pros and cons of AI-driven writing tools, please start a new topic.
Simon knows what Godwin’s law is. He just doubts (“?”) that his post had anything to do with that.