ok so this is difficult to explain: the difference between consciousness and whatever LLMs are doing is that computers lack the experience that living beings make. the reason why LLMs will never be reliable, i.e. not make grave mistakes, is because they lack a concept of responsibility, i.e. they are careless because there’s nothing on the line for them. they don’t know what it’s like to make a mistake and be wiped out from existence, trained over billions of years to not make these mistakes. meanwhile, living beings do. and that’s the difference.
You must log in or register to comment.
It’s mainly a category error. “experience”, “responsibility”, “careless”, “know” are all words for living, thinking entities. Sure, we like to antropomorphize AI. But at the end of the day there’s no “learning” or “experiencing” in this mathematical process. ChatGPT forgets everything the moment you close the chat because there’s nothing there to store what happened. It also doesn’t experience time or anything. Next time you open it, it’s again in exactly the same state as it always is. (They might tell it what time and date it is, though.)
AI also isn’t careless nor responsible. It’s a text generator. It just has one goal, and that is mimick human-like text. In a certain way. An we did some clever set-up so it prefers giving correct answers over giving wrong answers. But there’s no goal hardwired into AI like pleasing the user, or self-preservation. That’s just us anthropomorphizing it, because it generates text that sounds like it came from a human. Including all the story tropes.
But at the end of the day there’s no “experiencing” in this mathematical process.
How could you know? Consciousness is entirely a subjective experience. There’s no evidence of it anywhere else in the universe, so it seems baseless to make absolute claims about one thing being conscious and another not. You could reduce what the human brain does to something akin to a mathematical process too - it’s just neurons firing in the correct order. To me it seems that the only thing I can be certain of is that I’m conscious myself - whatever that self is - but I can’t make the same claim even of other people. I assume they’re conscious like me and they claim to be but I can’t be absolutely sure of it.
Consciousness comes without any good scientific definition. We don’t really know what it’s supposed to be. So it’s hard to talk about it in a scientific way. We can make some assumptions and go with that. For example I can learn in context. Remember what happened and what I do in 3 minutes will factor that in. A bare LLM simply doesn’t have that, it only has its context and no further “state of mind”. It can’t learn or change. So I’d say if that’s a requirement for “consciousness”, LLMs won’t have it. And furthermore it doesn’t have goals like I do. I have some evolutionary drive / goals to feel happy, procreate, whatever… A LLM also has a goal. But that’s to predict the next token so it generates text close to its training data. That’s very dissimilar, so I don’t see any reason to postulate we’re similar in a way. I mean it doesn’t rule it out either. But It’d be a big surprise to me if that dissimilar things end up with the same attributes. I mean for all I know a stone or prime numbers could have a consciousness and it’s hidden somewhere in the sixth dimension. Heck, maybe even God exists. But I don’t see any reason to believe in those things unless there’s some proof for big (and weird) claims like those.
I don’t think memory is a requirement for experience or emotion. I experience all kinds of emotional stuff when I’m dreaming, and I instantly forget the vast majority of it. So I think an AI could have emotions without memory.
My dreams also have zero stakes, and I full well know that. You can say I don’t know that because I’m asleep, but what about video games, or scary movies? No, the human brain will adapt its emotions to the stakes in the present context, even if they’re tiny. Case in point: toddlers. Dropping a cookie on the floor could literally be the worst thing that ever happened to them, and provoke a full on screaming meltdown. So I think an AI could experience strong emotions in response to very mild training incentives.
We also have experimental data showing that LLMs work harder when you threaten them. Maybe they’re just copying the human tendency to work harder when threatened, but I wouldn’t put money on it.
What I’m saying is, we have no idea, and I haven’t heard any convincing arguments either way.
Well, to be realistic, you probably don’t know whether your dreams have stakes if you can’t remember them?! I sometimes remember my dreams for a while. And the emotions feel completely realistic to me. Sometimes it affects my body and I’ll wake up from the emotions associated with a nightmare. Sometimes my wife pokes me because my legs move. I can’t remember most of them. Maybe one every other week, and just for a brief period. But I’d say they’re very realistic. I’ll also have positive dreams, or process what I experienced the time before and the dream will take action in the consequences of that. And the process dreaming is a part of, definitely changes my state. Before, I’m tired and depleted of energy, and after that I (hopefully) changed to being not tired and full of energy again. (At least after a decent amount of coffee.) But it definitely has an effect on me.
I recently saw some documentary about a woman with a 7 second memory. And I’d say experiencing things and having a memory, or experiencing things and not being able to remember them are two entirely different pair of shoes. She scribbles everything down to grab on to it and make it mean something. And if she reads it later, she has no recollection if it happening to her, nor any attached emotions. I’d say experiences without memory are a bit like having a J.R.R. Tolkien book in your bookshelf, which you didn’t read. It’s full of memories, processed, real information of the author. But what does that mean to you since you didn’t read it? I mean Tolkien certainly did exist. We can tell. But other than that, I don’t think his existence translates to anything in you, unless it also has some kind of effect on you?!
We also have experimental data showing that LLMs work harder when you threaten them. Maybe they’re just copying the human tendency to work harder when threatened, but I wouldn’t put money on it.
Also, when you dial down their capacity to deceive they start telling you they’re consciouss.
By consciousness I mean the fact of experience - that it is like something to be, that experience has qualia.
Things like memory, morality, biological goals, learning, intelligence, etc. that you listed there may be related, but they don’t need to be. It’s conceivable that we could create a very simple system that doesn’t seem to do much but by happenstance it processes information in a way that some level of consciousness emerges and it is like something to be that system. In the worst case we remain completely unaware of that and this thing is suffering immensely while we just keep “mistreating” it because we don’t know better. We effectively created hell and populated it.
Yeah I mean hypothetically… A lot of things are possible. It’s just always the question whether they’re part of reality.
And I need some way how to handle things. For example I’m fairly sure the apple tree and the lawn in my backyard are alive. I sometimes use sharp rotating knifes to cut down the lawn. I’ll also cut parts off of the tree, eat it’s fruits… Does it have an effect on those living organisms? Sure. They even react to their surroundings, the seasons, dryness. The tree reacts to branches being removed. But for some reason we thing it’s alright to mess with them. But for some reasons we don’t think it’s okay to cut off limbs off of the bird that sometimes sits on the tree… There seems to be some differentiating factor(s), even amongst living entities.
I can’t define consciousness but I am not spiritual in any way and I beleive it arises from some physical process. If a computer can be conscious it would need to have an analogous process somewhere in it’s hardware. This would mean that consciouness would not be limited to LLMs, and the mathematical process in, for example, excel, could also be argued to be consciousness. Does that make sense?
GenAI uses artificial neural networks, inspired by the human brain. We don’t know what characteristics of the brain cause experience, so we don’t know if we copied those characteristics when we copied human language abilities.
Reminder for some downvoters:
How voting works:
Vote the opposite of the norm.
If you agree that the opinion is unpopular give it an arrow up. If it’s something that’s widely accepted, give it an arrow down.It’s just fancy overvalued autocomplete.
Humans are consciouss, yet we too confidently spread false information constantly and it would be just as naive to blindly trust information only because it’s coming from a human.
Also, we can’t measure consciousness so claiming that LLM either is or isn’t consciouss is equally baseless - we simply can’t know and are only making assumptions that could one day turn out to be false.
They lack consciousness to begin with but yes, if we somehow managed to create something conscious and free-willed we’d still have to figure out how to inject prosociality, morality and responsibility into them. People are born with the capacity and natural predisposition towards it, and through propaganda, trauma and warped ideologies these attributes can be dimmed but it wouldn’t be the same for this hypothetical virtual being.
