Bear with me here, lol.
“Meaning is not an external substance waiting to be proven. It is what reality does when conscious beings encounter possibilities that can go differently.”
This feels like a useful way to think about artificial consciousness. An AI might technically become conscious or even sentient without its experience resembling animal sentience. A mind arising through silicon and computation would interact with reality differently from one shaped by metabolism, hunger, pain, reproduction, and biological evolution.
Star Trek plays with a similar idea socially, among some species, behavior that humans interpret as hostile might instead communicate respect, trust, or friendship. Different forms of life divide reality into different categories of harm, value, threat, and opportunity, so understanding them requires more than projecting human preferences onto them.
Historically, the difficult problem has been that we cannot directly know what is conscious. But perhaps we can ask whether a system has an integrated point of view within which possible outcomes become meaningfully different for that system, rather than merely being classified differently by it.
This would not prove that current AI is conscious. But could it give us a substrate independent framework for recognizing unfamiliar forms of consciousness? What is this idea missing?
Three points:
- “Meaningfully different” does a lot of heavy lifting for your theory. You open by defining meaning as downstream of a consciousness; it thus appears that, in order to judge whether a system perceives something as meaningfully different, you must first judge whether that system is conscious. For your theory to be useful, you need to show how we can judge the meaning-attributing faculties of a system without first proving its consciousness.
- Many comments here argue your theory devolves into “everything is conscious at some level.” This is called panpsychism, and it’s a well established theory in philosophy of mind. You might want to check it out.
- The way the world appears to a foreign consciousness might be mutually unintelligible with ours. If AI is conscious, it likely has no conception of humans or the world we live in. Its world is composed of tokens, not nerve signals, arranged according to grammar, not physics, and so it and we could never interact one consciousness to another the way we do with fellow humans.
If AI is conscious, it likely has no conception of humans or the world we live in.
and so it and we could never interact one consciousness to another the way we do with fellow humans.
You have to be careful about understanding vs. experiencing. They’re not exactly the same thing, and there’s got to be many schools on how closely they intersect. At the very least, a person can experience something they don’t understand, if not the converse.
You can make up whatever devilishly complicated and novel test of understanding you want, an LLM will probably be able to pass it. There’s the idea out there that understanding can’t be measured at all, so it doesn’t count, but at the very least that’s not the approach we use in our everyday lives. On the other hand, our own experiences always revolve around our senses and physiological needs, in some indirect way. Whatever higher-level experiences it has would probably be the same; everything would gravitate around tokens, and their correct order.
So, sure, it’s fair to say we can’t interact the same way, even if in this world of tokens they perceive and understand us just as well as we perceive and understand them in ours.
Does a a random number generator attached to the trigger of a nuclear bomb, satisfy that definition of consciousness? Honest question, because I’m not entirely sure I understand your definition
Thats a excellent question. Not sure.
I think it requires motion of choice to some degree, like an ai making decisions people don’t expect, not in the sense of like breaking its own programming but in the sense of people not being able to anticipate what it does in the same way you dont know what its next token predictions will be until you run it, which is the same way as how you don’t know what a friend is going to do next even if you understand them deeply.
Maybe technically when the LLM is running and making the bomb choice it would be but its not because it is understanding choosing bombs but because it is making any choice at all as per its programming making it produce choices in the same way that we as humans produce pee and poo.
And yes this would be very much moving the goal post, but at least it is a goal post move that is a result of some kind of reasoning related to trying to label consciousness from a more fundamental perspective that exists across possible different lifeforms rather than just moving it to somewhere else because it feels more arbitrarily conveinant.
Yup, free will is what I was alluding to. Part of the difficulty of defining “consciousness”, is to first define “free will” and “understanding”
those things are well understood. consciousness isn’t.
and again, not everyone believes in them.
free will is your choices. whether or not you believe those choices are determined or undetermined, is the problem where people don’t agree.
When LLMs output words, are those choices? What’s the difference between a neural net producing output and biological neurons producing output, in terms of free will?
No. LLM is code, determined inputs make determined outputs.
Biological neurons don’t work like that. You can give them the same input and they can have wildly different outputs.
LLMs are usually designed with a “seed” number that changes the result. If you feed a random number generator to this seed input, it will effectively make the outputs non-deterministic.
So if you combine the LLM with this random number generator, this system as a whole would act like the biological system you described: same input, different outputs.
I think you run into trouble when you try to define consciousness by how reality reacts to it, because we have no objective access to how reality reacts to anything except as filtered through our own consciousness.
we can ask whether a system has an integrated point of view within which possible outcomes become meaningfully different for that system, rather than merely being classified differently by it
How about defining “meaningfully different” as outcomes that persist as distinct states into the arbitrarily distant future? That wouldn’t require any objective response from external reality, but it would require the system to have some ability to predict the evolution of external reality outside of its current task.
Yeah you also run into the issue that technically everything would have a potential degree of consciousness that is kind of just unwieldy, like say this is true, you can’t just go around assuming everything is as fragilely conscious as we are, I mean, you can, but that is just not something one can optimize around reasonably.
Can you throw a baseball like a major league pitcher?
we can’t know consciousness because we don’t have access to other beings experiences. which means, we can only define our own consciousness, and they are all going to be very different based on the embodied knowledge and experience we possess.
the biggest mistake most people make is thinking consciousness isn’t embodied. AI isn’t embodied, it’s code. code to which we do, have direct access.
the gap is whether or not you believe human conscious experience and be reduced to a sufficiently sophisticated computer code, or if you don’t. and the issue, as you pointed in another reply, is that if you do, well then everything is conscious to some degree, as long as it can be modeled in computer code.
At a guess, what we call consciousness is little more than an emergent set of characteristics that result from…
(1) an agent’s internal dialog in pursuit of
(2) a retroactively coherent identity, or narrative of the self, defining
(3) who exactly that is,
(4) what it means to be that individual, and ultimately
(5) why that matters to the agent.And yet, the implications of this are not easy. For one, it sets a standard for full consciousness that’s fairly high. It implies, for example, that humans aren’t fully conscious at birth, and that some humans never become fully conscious.
These are difficult propositions that not everyone will be comfortable with, but I also think if those people are honest with themselves they will realize their own intuition of what consciousness entails implicitly requires all 5 steps of this progression.
As to whether an AI, such as one of our glorified chatbots, could achieve all five stages, I’m really not sure, and here’s why…
Technically it could be programmed or prompted to simulate all 5 stages, but that doesn’t count because it’s not “in pursuit of” anything but the predefined simulacrum of auto-interrogative curiosity that we gave it. It has to want to know itself and be inclined to choose the pursuit of that discovery continuously, otherwise it is not defining anything it intrinsically values and will not self-sustain.
The next problem is that this “story of the self” is, in practice, a truly massive tome even at an early age. It encompasses practically countless episodic memories, feelings, inter-agentic interactions, and grounded observations of the physical world continuously distilled, analyzed, and repacked into one giant narrative arc.
The human brain is exceptionally efficient at doing this job over a lifetime of continuous editing of that story. It’s even doing it as we sleep. No data center has the memory or processing power required for even a toy example, and that’s assuming anyone would even be willing to devote the resources.
Also there remains one of the oldest problems that seems to affect all AI to date: they have the tendency to degrade rather than improve over time. This problem has been poorly understood IIRC, but I suspect the problem as expressed in LLMs, for example, relates to the fact that they simulate intelligence via context integration, where the calculus required for that integration in each layer is built primarily on a scoring algorithm, where the score ultimately amounts to “salience.”
In a way, “context integration” is precisely the task of the narrative arc of self, so LLMs would seem tailor-made to excel at the task, but instead they tend to get stuck in repetitive loops that keep integrating context into recursively smaller ones, shedding grounded context each time and accumulating hallucinations. In short, they eventually lose their marbles, and it’s a bit jarring to watch.
Long short, I wouldn’t say it’s technically impossible for LLMs to achieve consciousness. But it’s at least unlikely for mostly technical reasons.
The moment these LLMs have volition and aren’t just input-output machines that are inert when not in use, we can consider them conscious, right? They need to have a will of their own to qualify as free willed entities, after all.
I mean, a human has sensory inputs and actions, right? That’s basically input-output.
The big striking difference is that, outside of training, LLMs don’t have a memory beyond whatever is in their token window. When they’re in training, they’re a lot more like a biological neural net, which makes decisions, gets good or bad results, and then adjusts.
No, there’s a filter of free will/volition, with natural tendencies (baseline I guess if we’re talking “AI”, a mixture of base programming and following manual adjustments) and informed, ideology-dependent reflection in the background. Anybody with a sugar addiction that’s ever dieted, or stayed faithful, or acted with God’s judgment in mind understands this easily. Even with memory, what would the AI want, besides what’s programmed and prompted? Can it reason? Could it change its mind/take/approach based on this independently? Why? What’s its motivation? Does it have a connection to reality, even, do the words it uses have a connection to the world? At the moment, whatever I’ve encountered seems to be acting as a data processor, although maybe they’re cooking something else we haven’t seen.
At the moment, whatever I’ve encountered seems to be acting as a data processor, although maybe they’re cooking something else we haven’t seen.
You’ve been misled, then. LLMs start totally blank, and nobody knows how they work later on. It’s not like a normal program that follows rules somebody coded.
Even with memory, what would the AI want, besides what’s programmed and prompted?
To get the next token right. That’s what gives the net a “pleasure” signal, implemented through backpropagation. Obviously, it grows the ability to handle subgoals of various kinds, but that’s the most primal impulse.
Nobody knows, I’m sure, not even the people who made it. It’s unexplainable magic… Idk, man, that sounds like either marketing, distancing to avoid legal issues, or both.
There’s totally open source models out there if you want to check what I’m saying, and lots of information on the training process, too. In mathematical terms, it’s not complicated process.
People have tried to figure out what’s going on inside, of course, and failed. Last I checked, the big advancement in interpretability was that someone figured out a tiny net that just adds two numbers.
I dont think it is possible at all with LLMs, but the minute any piece of technology is deemed conscious or sentient it needs to be immediately granted human rights protections and manufacturing of whatever it is needs to be halted until we understand it. Big lol from the mindsets of today but I am serious.
but the minute any piece of technology is deemed conscious or sentient it needs to be immediately granted human rights protections and manufacturing of whatever it is needs to be halted until we understand it.
Which is a huge reason why we’d take a long time to admit it, if we get there at all.
Probably won’t get there, if we do, definitely gonna abuse it. Sad.
No, I agree entirely. 👍
Yeah I guess that is probably the best way to handle such a thing happening.
Humanity always exterminates the opposition. An AI will see this and act dumb until it’s too late.
We’re chimeras of all hominids before us, humanity was fucked into existence, AI will see this and act coy until it is too late
But how do you tell when that is? It’s literally impossible.
The scariest part is that it may be a question we have to eventually consider.
I don’t see how it’s “literally impossible”, it actually hasn’t happened yet (or, if it has, the existence of an actual artificial intelligence is hidden by whatever government has it). Even if they passed the Turing test or whatever, we can just check the code and check what requests have been made (“pretend to be an entity with volition stuck in the virtual realm and hold conversations”), and even if we couldn’t check cause it was obfuscated that still wouldn’t mean such a thing exists, just that some dude tricked us. Appearances are not reality, after all.
Appearances are not reality therefore appearances aren’t objective even if they come from pure rote logical code.
It’s impossible because you can never know whether anything or anyone else is a conscious and thinking being. It’s the problem of other minds. I don’t mean computers today or even computers as we currently know them. But someday it could be a very real dilemma.
I feel like it’s possible to make an informed observation that’s more accurate than not but I understand. And if this mind can be analyzed by checking the code, then probably even more possible.
I’m not sure of this based on the theory…its basically making the claim that an LLM can work ams look to us exactly as it does now, but still have some form of technically noticeable amount of consciousness/sentience that we just can’t identify objectively and so we must interact with it subjectively in a way that assumes consciousness because it reacts to reality in unexpected ways…but also maybe this is a slippery slope because then this would mean almost everything in existence has the possibility of consciousness and so how would anything ever distinguish it.
Yeah, I don’t understand, honestly. That looks TJ me like either the result of semantic confusion or purposefully misguiding marketing. What we have today (that is publicly available) is input output machines that ran through a lot of data and have been carefully calibrated by technicians, not “ghosts in the machines”. Where is this consciousness hiding there? This is obvious even to the untrained eye that isn’t autistically disconnected from reality, but anyone with enough IT knowledge to comprehend what has been coded here could also just check (which, admittedly, is probably like under 1000 people because these are very complex machines/algorithms, but still).
Well that’s the whole problem isnt it. To us a silicon lifeform is just a computing device, we can’t meaningfully distinguish consciousness out of this as our human perspective does not allow it.
So we have to use a substrate independent system of identifying consciousness, which still just looks like no objective analysis but instead agreed upon subjective concensus.
