AI person reporting in. Without saying whether or not I personally believe that the current tools will lead to the end of humanity, I’ll point out a few possibilities that I find concerning about what’s going on:
The hype around AI is being used to justify mass layoffs, where humans are being replaced by tools that do a questionable job and can’t really understand the things those humans could understand. Whether or not the AI can do as good of a job according to some statical measurement is less relevant than the fact that a human is less likely to make an extremely grave mistake and more likely to be able to recognize when that does happen. I’m concerned this will lead to cross-industry enshitification on an unprecedented scale.
The foundation models consume a huge amount of energy. The more impressive you want it to be, the more energy it needs. As long as the data centers which run them are dependent on fossil fuels, they’ll be pumping a huge amount of carbon in the air just to do replace jobs that we didn’t need to have replaced.
As these tools are used more and more, they’re going to end up “learning” from content created by themselves instead of something that’s closer to a ground truth. It’s hard to predict what kind of degradation of service will come from this, but the more we create systems that rely on these tools, the more harm it will do to us.
Given the cost and nature of these tools, they’re likely to yield the most benefit to moneyed interests that want to automate the systems that maintain their power and wealth. E.g. generating large amounts of convincing disinformation to manipulate the public into supporting politicians or policies that benefit a small number of wealthy people in the short term while locking humanity into a path towards destruction.
And none of this accounts for possible future iterations of AI tools that may be far more capable than what exists today. That future technology will most likely be controlled by powerful people who are primarily interested in using it to bolster the systems that keep them in power, to the detriment of humanity as a whole.
Personally I’m far less concerned about a malicious AI intentionally doing harm to humanity than AI being used as a weapon by unscrupulous people.
I agree with everything you said and wanted to point out that you offered quite a compelling argument that even current AI tools are capable of significant amounts of damage without even touching on the autonomous weapons systems that are starting to be deployed.
Not even just talking about the military intelligence systems that may or may not have been deployed (Israel: Lavender et al), but we’re starting to show off weapons platforms that may someday be empowered to perform their own threat analysis and take real world actions accordingly. That shit is terrifying in more of a Terminator/Matrix way than anything else imo.
It’s good for marketing, though. “Ah, our software is so powerful, it could destroy humanity! Please pass a bill saying so while we market friendly chatbots to the public while actually making money by selling our products to despots and warmongers that might actually end humanity.”
It’s regulatory capture. Add deluded barriers to entry to make it difficult for competition and community projects to develop, and you have a monopoly.
Sure, but this outcome is not at all surprising. There are plenty of smart AI people that have nuanced views of what kind of threat could be posed by recklessly unleashing tools that we don’t fully understand into the hands of people who are likely to do harmful things with them.
It’s not surprising that those valid nuanced concerns get translated into overly simplistic misrepresentations entangled with pop sci fi panic around rogue AI as they try to move into public discourse.
We do fully understand them. Not knowing the exact reason they come to a model doesn’t mean the algorithm has a shred of mystery involved. It’s like saying we don’t understand fluid dynamics because it’s computationally heavy.
It’s autocomplete with a really big training set and a really big model. It cannot possibly develop agency. It’s hundreds of orders of magnitude of complexity short of a human.
That’s not what an algorithms researcher means when we talk about “understanding”. Obviously we know the mechanism by which it operates, it’s not an unknown alien technology that dropped into our laps.
Understanding an algorithm means being able to predict the characteristics of its outputs based on the characteristics of its inputs. E.g. will it give an optimal solution to a problem that we pose? Will its response satisfy certain constraints or fall within certain bounds?
Figuring this stuff out for foundation models is an active area of research, and the absence of this predictability is an enormous safety concern for any use cases where the output can be consequential.
It cannot possibly develop agency.
I don’t believe I’ve suggested anywhere that I think it will, but I’ll play around with this concern anyway… There’s a lot of discussion going on about having models feed back on themselves to learn from their own output. I don’t find it all that hard to imagine that something we could reasonably consider self awareness could be formed by a very complex neural network that is able to consume and process its own outputs. And once self awareness starts to form, it’s not that hard for me to imagine a sense of agency following. I have no idea what the model might use that agency for, but I don’t think it’s all that far fetched to consider the possibility of it happening.
There are plenty of nondeterministic algorithms. It’s not a special trait. There are plenty of algorithms with actual emergent behavior, which LLMs don’t have to any meaningful extent. We absolutely understand how LLMs work
The answer to both of your questions is not some unsolved mystery. It’s “of course not”. That’s not what they do and fundamentally requires a much more complex architecture to even approach.
Non-deterministic algorithms such as Monte Carlo methods or simulated annealing can still be constrained to an acceptable state space. How to do this effectively for LLMs is a very open question, largely because the state space of the problems that they are applied to is incomprehensibly huge.
It’s only an “open question” if you are somehow confused by the fact that it’s a super simple algorithm that cannot ever possibly be used like that.
It may be a small part of a proper architecture for a functional solution, but there’s no possibility that it will ever be doing the heavy lifting. It is what it is, and that’s an obvious dead end.
The hype around AI is being used to justify mass layoffs,
Ban stock buybacks, abolish non-competes, fine the CEO and major stockholders personally for layoffs.
The foundation models consume a huge amount of energy. The more impressive you want it to be,
Nuclear power, renewables, carbon tax
As these tools are used more and more, they’re going to end up “learning” from content created by themselves instead of something that’s closer to a ground truth.
Not really our problem it is their problem.
Given the cost and nature of these tools, they’re likely to yield the most benefit to moneyed interests that want to automate the systems that maintain their power and wealth.
Restore the fairness doctrine limit the ability of groups like Sinclair.
Got any other impossible to solve issues let me know.
I never suggested these problems are impossible to solve, but you haven’t solved them in your post because you haven’t laid out how to overcome the political and economic resistance to implementing any of this, and that’s where the biggest challenge is.
Although I think it’s naive to believe that nuclear power and renewable energy can allow us to keep consuming energy recklessly. Renewable energy technology still puts a significant strain on the environment, in terms of mining rare-earth elements, pollution produced during manufacturing, and material waste from devices that have reached end of life. Nuclear energy is rife with controversy… I used to be firmly in support of it, but I’ve grown skeptical, largely because of the ecological damage from the mining and construction processes, and we don’t have a clear story of what end of life looks like for a nuclear power plant. A plant can only be expected to operate for 40-60 years at which point it needs to be demolished and rebuilt, repeating the massive costs of material waste and construction all over again.
At the end of the day the only way for humanity to survive is for everyone to be reducing their consumption, but I honestly the think the vast majority of people today would rather die and take everyone else down with them than accept more responsible consumption habits.
you haven’t laid out how to overcome the political and economic resistance to implementing any of this, and that’s where the biggest challenge is.
That isn’t my job.
Although I think it’s naive to believe that nuclear power and renewable energy can allow us to keep consuming energy recklessly.
Misn showing me where I said that? Cause I am pretty freaken sure I mentioned a carbon tax and incentives for companies to generate their own power.
Renewable energy technology still puts a significant strain on the environment, in terms of mining rare-earth elements, pollution produced during manufacturing, and material waste from devices that have reached end of life.
More tech. Let me know when you have an actual challenge.
Nuclear energy is rife with controversy…
So are vaccines.
A plant can only be expected to operate for 40-60 years at which point it needs to be demolished and rebuilt, repeating the massive costs of material waste and construction all over again.
Are you being serious right now? I am in infrastructure and 40 years is well beyond the scope of anything I build. Get me a freaken Bible and I will swear on it, your waste system in your area can never ever ever last 4 decades. They are constantly having to rip it all apart and rebuild. 11 years is what I typically hope for. Find me a wet well that is 4 decades old, find me a pump, find me a screw conveyor, find me a metering pump, find me a shredder, find me the UPS/generator, find me a DCS, that lasts 40 years. I am pretty tempted to share your comment with the office tomorrow, so we can have a good laugh at it.
Now you compare that to nuclear. Where everything is overbuilt everything is accounted for. No one improvises. Stuff in nuclear plants outlasts everything else. I have worked on very non-critical systems for nuclear plants and had to follow the strictest rules of my career. It takes a certain level of insanity to specify what type of tape should be used on a bundle of wires.
Guys at nuclear plants are freaken artisans, unionized, paid the highest in the industry for a reason.
Got to love this site sometimes. No where else can I hear people arguing against highly trained people getting paid very well being evil and instead being told that everything was so freaken perfect during the dark ages.
So then you didn’t “solve” the impossible problems.
I hope you and your colleagues have a good laugh about how the work you do is contributing to the march towards the end of human civilization as we know it.
So then you didn’t “solve” the impossible problems.
I did. Let me know which part was confusing to you. Unless of course you want to choose to, yet again, latch on to one sentence as a gotcha.
end of human civilization as we know it.
Is that what we are doing? I thought I built recycling systems, sanitation systems, and pumping systems. I wasn’t aware that helping make sure we don’t die in our own feces+garbage and providing fresh water was going to be the downfall of civilization. Damn here I am thinking that this is one of the most important parts of civilization. Well I don’t want to cause civilization to end. Tell you what, why not be the change you want to see in the world and stop flushing your toilet, stop using tap water, stop recycling anything, and don’t set your garbage out.
The part where you left out any viable path for any of the hypothetical solutions to be realized 🤷♂️ You of all people should know that a blueprint is worthless if there’s no process available to build what it describes.
Damn here I am thinking that this is one of the most important parts of civilization.
I mean yeah, I do agree that sanitation and water works are the crowning achievement of human civilization to this very day. But I’ve gotta say it doesn’t inspire confidence if the people running those systems think that concerns about sustainability are something to have a group chuckle about.
Just because the work you do is important doesn’t mean it’s beyond the scrutiny of ecological sustainability. All your good work won’t amount to much in the long run if we can’t find a path to reducing consumption and prolonging the viability of these systems. We don’t have infinite resources, and our ability to recycle is nowhere near what it needs to be to keep up with economic demand.
Tell you what, why not be the change you want to see in the world and stop flushing your toilet, stop using tap water, stop recycling anything, and don’t set your garbage out.
My partner and I are unironically taking the time to research subsistence farming and how to maintain very basic personal water collection and waste removal/reuse systems. We’re also learning about perma-computing so that hopefully we can preserve some of the knowledge that humans have accumulated into the future.
We see it as a foregone conclusion that human civilization as we know it will entirely collapse, probably sooner than anyone cares to admit, so we’re making contingency plans. People with your dismissive attitude are a big part of why we see it as a forgone conclusion. Because as far as we can tell you’re in the 95%+ majority of people on this planet, which means hardly anyone is putting effort into solving these existential problems that we’re facing. Problems which you have offered no viable solution to, despite your insistence otherwise.
The part where you left out any viable path for any of the hypothetical solutions to be realized 🤷♂️ You of all people should know that a blueprint is worthless if there’s no process available to build what it describes.
Again. Not my job to handhold you. I build projects other people have the job of convincing the general public. Marketing is not engineering.
But I’ve gotta say it doesn’t inspire confidence if the people running those systems think that concerns about sustainability are something to have a group chuckle about.
You are the one who thought 6 decades was a reasonable number for moving fluid systems. Yes it is ridiculous. You can’t get parts that work that way no matter how much money you have. These aren’t items you can buy. I don’t even know how it would even work. How many bushings and filters and gaskets that need to have the exact dimensions they have and you expect them to last six decades. Tell me how to do that. How do you make something of millimeter thickness that is also rugged enough? Fuck even like paint, you can’t expect a paint job to last anywhere near that.
And even if you could build something like that I wouldn’t want to go anywhere near it. I hate dealing with the electrical systems from the 80s let alone from the 60s. That would be a nightmare. No way I am sticking my face inside a panel that predates SCCR.
My partner and I are unironically taking the time to research subsistence farming and how to maintain very basic personal water collection and waste removal/reuse systems
Alright have fun with that. It won’t be as inefficient or as safe but you are welcome to try.
Because as far as we can tell you’re in the 95%+ majority of people on this planet, which means hardly anyone is putting effort into solving these existential problems that we’re facing.
Literally this weekend answered an email about a battery recycling facility I am helping to design. But yeah your compost pile is so impressive to me.
I made a decision 8 years ago to go into waste and pollution reduction. I choose to be the change I wanted to see in the world. You grew an heirloom turnip, I wrote the code running on +100 scrubbers. As an environmental expert I am sure you know what a scrubber is and totally don’t have to look at Wikipedia to know the difference between a hybrid/wet/dry types. As well as what blowoff, recirculation lines, mist eliminators, and reaction chambers are.
Problems which you have offered no viable solution to, despite your insistence otherwise.
Umm actually I did. Repeatedly. You just waved your hands around, pointed at the one sentence you thought you could attack, declared victory, then bragged about your little garden.
AI takes a crazy amount of power, which is largely fueled by the same fossil fuels that are indeed killing us off and destroying our habitat, which will kill even more of us, so AI could definitely indirectly kill off humanity.
No. There are not.
Believing anything resembling current tools has the capacity to end humanity in incontrovertible proof that you are not smart.
AI person reporting in. Without saying whether or not I personally believe that the current tools will lead to the end of humanity, I’ll point out a few possibilities that I find concerning about what’s going on:
The hype around AI is being used to justify mass layoffs, where humans are being replaced by tools that do a questionable job and can’t really understand the things those humans could understand. Whether or not the AI can do as good of a job according to some statical measurement is less relevant than the fact that a human is less likely to make an extremely grave mistake and more likely to be able to recognize when that does happen. I’m concerned this will lead to cross-industry enshitification on an unprecedented scale.
The foundation models consume a huge amount of energy. The more impressive you want it to be, the more energy it needs. As long as the data centers which run them are dependent on fossil fuels, they’ll be pumping a huge amount of carbon in the air just to do replace jobs that we didn’t need to have replaced.
As these tools are used more and more, they’re going to end up “learning” from content created by themselves instead of something that’s closer to a ground truth. It’s hard to predict what kind of degradation of service will come from this, but the more we create systems that rely on these tools, the more harm it will do to us.
Given the cost and nature of these tools, they’re likely to yield the most benefit to moneyed interests that want to automate the systems that maintain their power and wealth. E.g. generating large amounts of convincing disinformation to manipulate the public into supporting politicians or policies that benefit a small number of wealthy people in the short term while locking humanity into a path towards destruction.
And none of this accounts for possible future iterations of AI tools that may be far more capable than what exists today. That future technology will most likely be controlled by powerful people who are primarily interested in using it to bolster the systems that keep them in power, to the detriment of humanity as a whole.
Personally I’m far less concerned about a malicious AI intentionally doing harm to humanity than AI being used as a weapon by unscrupulous people.
I agree with everything you said and wanted to point out that you offered quite a compelling argument that even current AI tools are capable of significant amounts of damage without even touching on the autonomous weapons systems that are starting to be deployed.
Not even just talking about the military intelligence systems that may or may not have been deployed (Israel: Lavender et al), but we’re starting to show off weapons platforms that may someday be empowered to perform their own threat analysis and take real world actions accordingly. That shit is terrifying in more of a Terminator/Matrix way than anything else imo.
Zero of these things are impacted by this legislation in any way.
This is exclusively the mentally unstable “killer AI” nonsense. We’re not even 1% of 1% of the way to anything resembling agency.
It’s good for marketing, though. “Ah, our software is so powerful, it could destroy humanity! Please pass a bill saying so while we market friendly chatbots to the public while actually making money by selling our products to despots and warmongers that might actually end humanity.”
It’s regulatory capture. Add deluded barriers to entry to make it difficult for competition and community projects to develop, and you have a monopoly.
Sure, but this outcome is not at all surprising. There are plenty of smart AI people that have nuanced views of what kind of threat could be posed by recklessly unleashing tools that we don’t fully understand into the hands of people who are likely to do harmful things with them.
It’s not surprising that those valid nuanced concerns get translated into overly simplistic misrepresentations entangled with pop sci fi panic around rogue AI as they try to move into public discourse.
We do fully understand them. Not knowing the exact reason they come to a model doesn’t mean the algorithm has a shred of mystery involved. It’s like saying we don’t understand fluid dynamics because it’s computationally heavy.
It’s autocomplete with a really big training set and a really big model. It cannot possibly develop agency. It’s hundreds of orders of magnitude of complexity short of a human.
That’s not what an algorithms researcher means when we talk about “understanding”. Obviously we know the mechanism by which it operates, it’s not an unknown alien technology that dropped into our laps.
Understanding an algorithm means being able to predict the characteristics of its outputs based on the characteristics of its inputs. E.g. will it give an optimal solution to a problem that we pose? Will its response satisfy certain constraints or fall within certain bounds?
Figuring this stuff out for foundation models is an active area of research, and the absence of this predictability is an enormous safety concern for any use cases where the output can be consequential.
I don’t believe I’ve suggested anywhere that I think it will, but I’ll play around with this concern anyway… There’s a lot of discussion going on about having models feed back on themselves to learn from their own output. I don’t find it all that hard to imagine that something we could reasonably consider self awareness could be formed by a very complex neural network that is able to consume and process its own outputs. And once self awareness starts to form, it’s not that hard for me to imagine a sense of agency following. I have no idea what the model might use that agency for, but I don’t think it’s all that far fetched to consider the possibility of it happening.
There are plenty of nondeterministic algorithms. It’s not a special trait. There are plenty of algorithms with actual emergent behavior, which LLMs don’t have to any meaningful extent. We absolutely understand how LLMs work
The answer to both of your questions is not some unsolved mystery. It’s “of course not”. That’s not what they do and fundamentally requires a much more complex architecture to even approach.
Non-deterministic algorithms such as Monte Carlo methods or simulated annealing can still be constrained to an acceptable state space. How to do this effectively for LLMs is a very open question, largely because the state space of the problems that they are applied to is incomprehensibly huge.
It’s only an “open question” if you are somehow confused by the fact that it’s a super simple algorithm that cannot ever possibly be used like that.
It may be a small part of a proper architecture for a functional solution, but there’s no possibility that it will ever be doing the heavy lifting. It is what it is, and that’s an obvious dead end.
Ban stock buybacks, abolish non-competes, fine the CEO and major stockholders personally for layoffs.
Nuclear power, renewables, carbon tax
Not really our problem it is their problem.
Restore the fairness doctrine limit the ability of groups like Sinclair.
Got any other impossible to solve issues let me know.
I never suggested these problems are impossible to solve, but you haven’t solved them in your post because you haven’t laid out how to overcome the political and economic resistance to implementing any of this, and that’s where the biggest challenge is.
Although I think it’s naive to believe that nuclear power and renewable energy can allow us to keep consuming energy recklessly. Renewable energy technology still puts a significant strain on the environment, in terms of mining rare-earth elements, pollution produced during manufacturing, and material waste from devices that have reached end of life. Nuclear energy is rife with controversy… I used to be firmly in support of it, but I’ve grown skeptical, largely because of the ecological damage from the mining and construction processes, and we don’t have a clear story of what end of life looks like for a nuclear power plant. A plant can only be expected to operate for 40-60 years at which point it needs to be demolished and rebuilt, repeating the massive costs of material waste and construction all over again.
At the end of the day the only way for humanity to survive is for everyone to be reducing their consumption, but I honestly the think the vast majority of people today would rather die and take everyone else down with them than accept more responsible consumption habits.
That isn’t my job.
Misn showing me where I said that? Cause I am pretty freaken sure I mentioned a carbon tax and incentives for companies to generate their own power.
More tech. Let me know when you have an actual challenge.
So are vaccines.
Are you being serious right now? I am in infrastructure and 40 years is well beyond the scope of anything I build. Get me a freaken Bible and I will swear on it, your waste system in your area can never ever ever last 4 decades. They are constantly having to rip it all apart and rebuild. 11 years is what I typically hope for. Find me a wet well that is 4 decades old, find me a pump, find me a screw conveyor, find me a metering pump, find me a shredder, find me the UPS/generator, find me a DCS, that lasts 40 years. I am pretty tempted to share your comment with the office tomorrow, so we can have a good laugh at it.
Now you compare that to nuclear. Where everything is overbuilt everything is accounted for. No one improvises. Stuff in nuclear plants outlasts everything else. I have worked on very non-critical systems for nuclear plants and had to follow the strictest rules of my career. It takes a certain level of insanity to specify what type of tape should be used on a bundle of wires.
Guys at nuclear plants are freaken artisans, unionized, paid the highest in the industry for a reason.
Got to love this site sometimes. No where else can I hear people arguing against highly trained people getting paid very well being evil and instead being told that everything was so freaken perfect during the dark ages.
So then you didn’t “solve” the impossible problems.
I hope you and your colleagues have a good laugh about how the work you do is contributing to the march towards the end of human civilization as we know it.
I did. Let me know which part was confusing to you. Unless of course you want to choose to, yet again, latch on to one sentence as a gotcha.
Is that what we are doing? I thought I built recycling systems, sanitation systems, and pumping systems. I wasn’t aware that helping make sure we don’t die in our own feces+garbage and providing fresh water was going to be the downfall of civilization. Damn here I am thinking that this is one of the most important parts of civilization. Well I don’t want to cause civilization to end. Tell you what, why not be the change you want to see in the world and stop flushing your toilet, stop using tap water, stop recycling anything, and don’t set your garbage out.
The part where you left out any viable path for any of the hypothetical solutions to be realized 🤷♂️ You of all people should know that a blueprint is worthless if there’s no process available to build what it describes.
I mean yeah, I do agree that sanitation and water works are the crowning achievement of human civilization to this very day. But I’ve gotta say it doesn’t inspire confidence if the people running those systems think that concerns about sustainability are something to have a group chuckle about.
Just because the work you do is important doesn’t mean it’s beyond the scrutiny of ecological sustainability. All your good work won’t amount to much in the long run if we can’t find a path to reducing consumption and prolonging the viability of these systems. We don’t have infinite resources, and our ability to recycle is nowhere near what it needs to be to keep up with economic demand.
My partner and I are unironically taking the time to research subsistence farming and how to maintain very basic personal water collection and waste removal/reuse systems. We’re also learning about perma-computing so that hopefully we can preserve some of the knowledge that humans have accumulated into the future.
We see it as a foregone conclusion that human civilization as we know it will entirely collapse, probably sooner than anyone cares to admit, so we’re making contingency plans. People with your dismissive attitude are a big part of why we see it as a forgone conclusion. Because as far as we can tell you’re in the 95%+ majority of people on this planet, which means hardly anyone is putting effort into solving these existential problems that we’re facing. Problems which you have offered no viable solution to, despite your insistence otherwise.
Again. Not my job to handhold you. I build projects other people have the job of convincing the general public. Marketing is not engineering.
You are the one who thought 6 decades was a reasonable number for moving fluid systems. Yes it is ridiculous. You can’t get parts that work that way no matter how much money you have. These aren’t items you can buy. I don’t even know how it would even work. How many bushings and filters and gaskets that need to have the exact dimensions they have and you expect them to last six decades. Tell me how to do that. How do you make something of millimeter thickness that is also rugged enough? Fuck even like paint, you can’t expect a paint job to last anywhere near that.
And even if you could build something like that I wouldn’t want to go anywhere near it. I hate dealing with the electrical systems from the 80s let alone from the 60s. That would be a nightmare. No way I am sticking my face inside a panel that predates SCCR.
Alright have fun with that. It won’t be as inefficient or as safe but you are welcome to try.
Literally this weekend answered an email about a battery recycling facility I am helping to design. But yeah your compost pile is so impressive to me.
I made a decision 8 years ago to go into waste and pollution reduction. I choose to be the change I wanted to see in the world. You grew an heirloom turnip, I wrote the code running on +100 scrubbers. As an environmental expert I am sure you know what a scrubber is and totally don’t have to look at Wikipedia to know the difference between a hybrid/wet/dry types. As well as what blowoff, recirculation lines, mist eliminators, and reaction chambers are.
Umm actually I did. Repeatedly. You just waved your hands around, pointed at the one sentence you thought you could attack, declared victory, then bragged about your little garden.
If we get killed by auto complete we deserve to die.
AI takes a crazy amount of power, which is largely fueled by the same fossil fuels that are indeed killing us off and destroying our habitat, which will kill even more of us, so AI could definitely indirectly kill off humanity.