cross-posted from: https://lemmy.blahaj.zone/post/46091883
Though nobody expected it in the first place, AI is not perfect and can make mistakes (obviously 🙄) What’s your experience with this? I’d love to hear about it.
Work demanded we lean heavy into AI, bought into ChatGPT early
Our first townhall meeting after that was so cringe. They made a bunch of uncanny valley slides up with amazing numbers how they were ruling the industry, They even included pages of citations.
Every single citation was a 404. Half the numbers given for the rest of the industry were decimated versions of their real values.
They spent a lot of money on tokens to unknowingly make up a deck of lies, then presented them to thousands of people.
Talk about spreading misinformation
I think the garlish recipe that grew deadly botulism cultures is still the banger. Looks innocent and can kill your whole family (here the original reddit post - “Google Gemini tried to kill me”).
And it is a good example why being right 50% of the time is not good enough and definitely not safe even for very commonday things.
That’s what I don’t get about people who try to justify AI by saying ‘well it’s not always wrong’
It’s not about the fact that it’s not always wrong, it’s about the fact that it might very well bloody kill you if it gets something severely wrong
Suggesting that, since bleach is a great cleaner, and ammonia is a great cleaner, mixing the two would make an even better cleaner.
That can’t be true that AI suggested that of all things???
You can use it to clean stuff really really well. Once.
Not personal but the instances where AI tells people who are suicidal to go through with it or recommend people do things that ultimately kill them.
Truly awful behaviour from AI
Not exactly life-critical but on games I play I’ve asked for optimization advice and it has responded with info I know is 100% wrong. I end up replying with URLs to the info and pasting the direct info from it. The AI of course goes, “Of course!”.
At work I constantly get commands that dont work in my version of some app or OS even after I told it the exact version I’m using. I’ll even post the response from tech support and it’ll throw up its virtual hands and say “of course!”.
I’ve also noticed that AI has now been around long enough to start posting clearly outdated information.
I work in software development. At one of my jobs i had an elderly colleague with onset dementia, who was causing all kinds of problems.
As a last hurra before going into retirement he wanted to prove to everyone (or better, himself) that he could still do it and code a big application all by himself.
So he decided to take on this project where he would build an intranet portal for a customers internal website…with WordPress.
With the help of AI, he coded a plugin and a theme. Which in itself is not a problem, but he made them depend hard on each other. There were constants defined in the theme that were used in the plugin and the AI did not add any checks, so when you disabled one of the two, the site would just throw an error 500.
When it went live, the customer noticed that only administrators were able to login, any user level lower than that (i.e. all of the customers employees) would just get an error on login.
By then the old fart had decided he was leaving the company, so I received the task to clean that up. He had used two different versions of an external library that were slightly incompatible with each other and depending on a race condition we would get one or the other, the LLM hooked into things that didn’t actually exist, functions were duplicated between the plugin and theme, the CSS was 20.000 lines. It was an absolute shitshow and basically unrecoverable.
The funniest thing was that this guy was absolutely proud of his work and anyone who just slightly criticized him for the atrocious code quality just got shouted at loudly.
My boss then hired someone fresh from university to fix that and he just put more AI code into the project, making it even worse.
When I left, the project was still not launched after over a year and the customer was (rightfully so) super pissed-off.
Think of all the money the company saved on employees though!
Needless to say, the customer will most likely not be hankering after AI-based development in the future I guess
That’s the one positive thing that came out of this project. :)
Fun fact: The project was also supposed to have a feature where the customer could add new pages with content and then decide where in the main menu those pages should appear.
WordPress internally uses a very basic system for assigning menu items - if you have ten menu items, they have an order that goes from 1 to 10. Number 1 is the first, 10 is the last - very simple (and stupid, because you can’t add numbers in between to add new menu items on the fly).
The old fart (or the AI) wrote a system where the customer would just put in a number for the menu order. So if they created a page and then set it to be the fourth item in the menu…there already was a fourth item and WordPress decided at random where to put the menu item instead. Sometimes it was in the correct place. Most of the time it landed somewhere unintended…
What you would actually have to do here is look at all of the menu items, shove the new item into that list and then recalculate all of the numbers and save them. AI was not able to do it, no matter how you prompted it. We tried Gemini, Claude Code, ChatGPT, all of them shit the bed completely.
So there we were - this feature absolutely had to be implemented by a human. I didn’t have the time for it, the old fart left and the student was completely useless without chatbots helping him out. That put the project on hold for six months until I was available again, but then I left. Whoopsie!
Fun fact #2: The company eventually went bankrupt because of projects like this and was bought out by a competitor.
1-2 punch the boss decided to save money by hiring a fresh grad likely used AI throughout his whole degree.
Seems a general thing, that AI is capable to generate LOTS of code with plausible-looking patterns, but incapable to understand and maintain any invariants or pre-conditions - not even the simplest ones.
Also matches nicely to dementia in that the code looks plausible and good on the local scale, but does not have a coherent global logic behind it - because that requires far more context, learning and actual logical reasoning, which these things can only simulate.
And is the competitor as dysfunctional as this was?
They’re a little more competent, but that doesn’t say much, given how extremely bad this project was run - it’s probably very hard to do worse.
Fair enough
Last year, 2025, Google’s Ai overview confidently stated that four pieces of 26 gauge wire twisted together would be the equivalent of one 32 gauge wire. For those that are unfamiliar with wire gauges, a higher gauge number means a smaller wire.
Technically it’s true it can carry the same (but more) current
I would say you’re wrong but you disagreed with yourself already so I’m not sure how to proceed.
Mind blowing. Plz never use AI for life things
Not too long ago, I was assigned Review on a clearly vibecoded PR. It was for a feature that parsed hourly values from a csv file and put them into the Database. Easy enough at first glance, just loop through the hours of the day and grab the value for each hour, with an error if your file doesn’t contain all the hours of the day. But for some reason my coworker not only decided to introduce a whole new “hour of the day” type to loop over instead of looping over numbers, but that type explicitly contained 25 values. It was the numbers 0 to 24. There was even a comment next to it saying that it had 25 values. All test cases were green too, because the test data was generated specifically to satisfy the tests. I had to explain to a grown ass adult that the day only has 24 hours and it wouldn’t work because there will never be a value for “24:00” in the real data. That specific coworker already made really stupid mistakes before AI was a thing, but at least those mistakes were easy enough to spot and didn’t secretly propagate themselves through thousands of lines of code.
The level of incompetence that AI can mask is absolutely stunning. (And this precisely is probably why the AI hype resembles so much Hans Christian Andersens Tale “The Emperor’s new Clothes”.)
I had to explain to a senior embedded software architect that you need to use locks to access and change the same variable in multiple threads in concurrent C++ code.
Make the variable a word size relevant to the processor’s memory access width (eg 32bit), don’t use atomic locks, mark it as volatile, write a comment about how things are perfect this way and lockers are losers; then rig the struct it’s in to have a 50% chance of misaligning the 32bits (so two cycles are required to read/write to it) every time someone changes and recompiles the code >:D
(Yes I no longer work as a programmer, no I don’t know why they don’t want me)
the test data was generated specifically to satisfy the tests
I once saw someone claim that slopcoded bugs would be prevented by the AI testing its code and tried to reason that no test from within the system could fully prevent errors within that same system. This is basically the kind of thing I had in mind: Manipulating the tests to fit the results instead of altering the code to fit the tests.
Testing has to come from outside, and with real data. Otherwise it’s just a meme of the AI giving itself a medal for passing its own tests.
AI giving itself a medal for passing its own tests
Sort of a Trump Turing Test
Please don’t tell me you’re not using FIFA compliant LLMs? That’s negligence!
That is now even augmented by AI writing all of specs, docs, and comments. I am not joking - I was told that in an interview.
Wow. Mind blowing that people actually behave like this (and think implementing AI in this circumstance is a good idea)
I had a lengthy discussion with another developer who was afraid some idea won’t work just because the AI he asked basically had no idea what he was talking about. Instead of asking me as the specialist in house for this niche topic.
Clearly it’s time for a rethink… (them, that is)
That time the US bombed a school and killed over a hundred children because AI said to target it
That … cannot be the reason we missile’d those kids. Is that the reason?
Around 2023, the early days of AI and Google shoving it in everyone’s faces in their search, I was trying to look up how to get nail glue off of a phone screen because my coworker at the time, during her shift, was doing her nails at the desk and got nail glue on the work phone.
The first result, Google’s AI summary, suggested to “microwave the phone for one to two minutes.” I clicked the source and it linked to a forum comment by a guy who was clearly joking about how to clean the phone, but the AI presented it as legitimate advice.
I clicked the source and it linked to a forum comment by a guy who was clearly joking about how to clean the phone, but the AI presented it as legitimate advice.
So AI thinks like a reddit mod? “This couldn’t possibly be a joke. Get the ban hammer.”
Wow. Fuck, that’s dumb even by AI standards (although it told me the other day I was, and I quote, “justified in wanting to implode”)
what was the context?
Basically I said I wanted to implode because of some frustration or something, and the way it started was ‘you are perfectly right to want to implode’ or something, rather than discourage it
Made a pr to change a thing after i told it specifically where and in which repo. It made the pr in wrong repo.
Unsurprising. AI told me the other day i was “right to want to implode” 🤨
Just got an AI generated PR from a coworker that implemented an entire microservice to replace the user id with username in one error message.
The kicker was there was already the username in the api method’s context. It was like 1500 lines, a new cache, a bunch of weird asynchronous thread code… and zero screenshot or proof it worked.
In the same vein, previous semester in Uni, we had a team project, not something too big, just very rushed.
At one point we were to contact a tcp server with JavaScript, send a one off measage, and return the reply, as the tcp server was a backing service.
My dumbass project partner has made a new server for it, just for being in the middle, so he wouldn’t have to check how to make a single tcp request using async, he fluffed it up calling it “The Middleware” like anyone was going to respect it, and it took him almost two weeks working on it.
Wish i could just take that task away from him but it was his ONE TASK, and i had many others, safe to say i had to retake that course. Which is honestly humiliating since that course barely thought me anything to begin with.
This semester the task popped up again, took me half an hour to have the gist of the task down, in 12 lines of untyped JavaScript, ~120 with JSDOC types and convenience functions, didn’t take much time at all.
Imagine wasting two weeks working your ass off because you can’t be arsed to learn a simple concept, I’m sure he was grooming his own ego with the AI about how smart his implementation was, it required an extra dockerfile, documentation for it, and testing was out of the question because the code was utter garbage.
That sucks, why did you have to take the whole course again over a shitty group project?
For what it’s worth, students have always been dumbasses in programming classes. We made a card game in one and one of my classmates had 52 if statements to get the card suit and value.
That’s probably nit nearly the most egregious, just one I remember.
The project was most of the grade, you need to pass the test on top of it, and the grades we got for the project were shite because the same guy was responsible for the maintenance of the dockerfile.
This semester i was responsible, and all pf his past tasks were trivial on top of my past ones, which i already remembered how to do, of course the project wasn’t entirely the same, just required the same things mostly.
A series of 52 if statements is plain irresponsible though lol.
The 2026 equivalent of paying outsourced developers by LOC is paying AI companies by the token
So it wasn’t broken… but they fixed it regardless and made it… worse?
This ticket was a small chore rather than critical bug. Just hygiene of keeping ids internal.
Literally all they needed to do was switch
user.idtouser.usernamein the result object.The user fields are always in context because they’re stored on the session, and even if they weren’t, there was a fetch on users they could be trivially added to a few lines up in the method. Even a second query in the method wouldn’t be a big deal, as it would add a couple ms at most.
We get username from another service, so it’s already clean. The AI seems to have not understood this and went insanely defensive.
Way worse code than the original, though I’m sure it probably passes all the new tests.
Edit: this is what reading the PR felt like https://www.youtube.com/watch?v=y8OnoxKotPQ
EDIT: Thanks to Iran’s refusal to allow reporters or international investigators to look at the debris of one school which was bombed recently killing about 170 children, we may never know with any certainty who was responsible, but it is highly likely that the USA bombed the site because the building was included in legacy documents and AI would likely not exclude such older intel.
Changhan Kim, CEO of the South Korean gaming company KRAFTON, decided he needed a way out of a costly acquisition deal, he asked ChatGPT.
It didn’t work.
Obviously this was only a $250 Million dollar mistake (and ruined what little reputation his company had) but if you want an example which results in potential loss of human life: UnitedHealthcare has spent the last few years automating claim denial.
The US Department of State has also been using AI to shut down federal programs which include the words “gender”, “binary”, and “inclusive” for being woke dei. As a result grants were denied for museum air conditioning and publishing mathematics papers.
Early adopters of AI included Airlines which resulted in lawsuits over refundable tickets, and drive-thrus resulting in the purchase of 9000x Water (free).
TBH, in my opinion, all LLM output is worthless slop, equally stupid, always the dumbest. The only way we quantify one being dumber than the other is by the idiocy of the user justifying their insane actions with it because it’s a machine which inherently causes psychosis in the dumb and feeble.
The US Department of State has also been using AI to shut down federal programs
And the state department is using AI for making their maps.
It is like the laziest, stupidest kids in your classes got in control.
God damn. I gotta follow up on that. With enough finangling I might get appointed Ambassador.
Well everyone knows the Ivory Coast is famously land locked.
The headliner about flock cameras getting 71 percent of the stolen tag alerts wrong. You will have better luck flipping a coin on every sus car you see
Link?
That is an appalling statistic
