- cross-posted to:
- aicompanions@lemmy.world
- technology@lemmy.ml
- cross-posted to:
- aicompanions@lemmy.world
- technology@lemmy.ml
Avram Piltch is the editor in chief of Tom’s Hardware, and he’s written a thoroughly researched article breaking down the promises and failures of LLM AIs.
You know, I think ChatGPT is way ahead of a toaster. Maybe it’s more like a small animal of some kind.
One could equally claim that the toaster was ahead, because it does something useful in the physical world. Hmm. Is a robot dog more alive than a Tamagotchi?
There are a lot of subjects where ChatGPT knows more than I do.
Does it know more than someone who has studied that subject their whole life? Of course not. But those people aren’t available to talk to me on a whim. ChatGPT is available, and it’s really useful. Far more useful than a toaster.
As long as you only use it for things where a mistake won’t be a problem - it’s a great tool. And you can also use it for “risky” decisions but take the information it gave you to an expert for verification before acting.
Sorry to break it to you, but it doesn’t “know” anything except what text is most likely to come after the text you just typed. It’s an autocomplete. A very sophisticated one, granted, but it has no notion of “fact” and no real understanding of the content of what it’s saying.
Saying that it knows what it’s spouting back to you is exactly what I warned against up above: anthropomorphization. People did this with ELIZA too, and it’s even more dangerous now than it was then.
By the same logic, living creatures like us don’t know anything, we’re just anticipating whichever actions will increase our probability of reproduction the most. We’ve learned a bunch of things in the process, though, or at least we usually assume we have.
You could argue that the contextual learning LLMs do is fundamentally different from the thinking we do, and some experts do hold that opinion, but probably more hold the opposite. To prove it you could to devise an experiment that trips them up, and such things have been devised, but usually the next generation of LLM passes them. Until we define what “real knowledge” is, exactly, it’s going to be hard to reach consensus on if they have it.
While I can’t define “real knowledge” in a way that would satisfy a philosopher, I do know that LLMs, as they currently exist, lack something significant in their ability to form connections.
If you invite an LLM to complete the sentence "I’m going to walk my ", one of the things it’s likely to tack on at the end is “dog”, but that isn’t because it understands that a “dog” is a mammalian quadruped often kept as a pet that requires exercise at intervals, it’s just because the next word in the body of text that it’s already consumed is more likely to be “dog” than anything else. It doesn’t understand why “refrigerator” and “hippopotamus” aren’t equally likely likely endings to the sentence, it only knows it doesn’t see them as often.
It doesn’t connect “three” with the number that you get by taking one thing and adding another and then one more. It doesn’t link “dusky rose” with a colour somewhere around #b1809b.
I guess what I’m saying is that it lacks an understanding of the world as a set of objects with intrinsic properties that can be divided into categories (which isn’t a complete description of how humans perceive the world either). Its world-model consists of words, which have no intrinsic link to the object they describe, and may indeed refer to very different things in different contexts (or, worse, different languages). A specimen of canis familiaris is a dog, but there are also sun dogs, corn dogs, bench dogs, etc, which might be referred to as simply “dogs” depending on context, and the LLM doesn’t know that they’re completely different from the animal.
Note that word “intrinsic”. It’s important. If I show a giraffe to a human who’s never seen one before, they may not have a word for it, but they can still determine things about it: it’s an animal, it has four legs, it has a long neck, it’s yellow-brown with darker spots (with only rare exceptions).
If I present the word “giraffe” to an LLM that doesn’t have it in its training data, it means nothing more to it than “quobble” would. Words (with the possible exception of some onomatopoeia) have no intrinsic link to the things they represent. The LLM can examine context and frequency (which are intrinsic properties of words in running text), but not meaning, because it has no model for meaning. Lacking that model, it has no actual knowledge (except about word frequency and context).
Actually, if you asked it, it would probably be able to formulate an explanation of that. It would do so as a form of text prediction, but the output would be original and correct anyway. How that’s different from a person answering you correctly so you’ll like them and won’t club them for mammoth meat is all philosophy.
What would it take for you to conclude that an LLM does understand meaning? Would it have to have a meaning subroutine written explicitly into it? How do you know there isn’t one, just in a form we can’t recognise, just as it’s so hard to see thoughts in our pink goo? You have an intuition here, and that’s valid, but the world is often unintuitive, and I’d urge you to suspend final judgement until we have things more nailed down.
That depends on the human though, doesn’t it? If it was a blind person, they could only understand it through it’s calls and it’s stink; it probably would be too dangerous and skittish to touch even. To really explain it, you’d still have to use language, and yet I think a blind person can understand a giraffe just fine.
This isn’t the first time it’s come down to how powerful language is on here. That seems to be the main point of divergence between skeptics and the more believer-ish camp.
Going over it, I think that sentience (not necessarily sapience, but that would be nice) is the secret sauce. In order for me to accept that an AI knows something (as opposed to posessing data which it does not actually understand), it has to demonstrate awareness.
So how can a text-based AI demonstrate awareness, given the constraints of the interface through which it must operate? Reliably generalizing from data not immediately part of the response to the current prompt might do it. Or demonstrating that it understands the consequences of its actions in the real world. Even just indicating that it knows when it’s making things up would be a good start.
For instance, take the case of the ChatGPT-generated fake legal citations. An AI which would have been fed masses of information relating to law (I’d expect that to include law school textbooks, from archive.org if nowhere else) demonstrated very clearly that it did not know that making up legal cases in response to a factual query was a Very Bad Idea. It did not generalize from data outside the domain of lists of case names that would have told it not to do that, or provide any indication that it knew its actions could be harmful. That AI had data, but not knowledge.
So we’re back to connections and conceptual models of the world again.
I mean, was it a bad idea? For the lawyer sure, but ChatGPT was not penalised by it’s own cost function. It may well have known in some way that is was just guessing, and that generally a legal document is serious business, but it doesn’t have any reason to care unless we build one in. Alignment is a whole other dimension to intelligence.
It sounds like the biggest models do this reasonably well. Commonsense reasoning would count, right?