The test seems kind of dogshit, you could make the same argument against any tool, calculators or even abacuses would have the same effect.
I’m made to use it for work and it does speed up some tasks, however for some stuff it ends up being like the experiment where not doing the work the first time means the whole process takes longer at the end.
To add to this, we already know that context switching causes a loss in performance.
A person who’s thinking about how to solve a problem one way and then has to suddenly think about solving it in another way will perform worse.
The Neuroscience Behind the Pain
Context switching isn’t just annoying — it’s neurologically expensive. When you shift from debugging a race condition to answering emails, your brain doesn’t simply “change tabs.” It goes through a complex process:
-Memory consolidation: Storing your current mental model
-Attention disengagement: Breaking focus from the current task
-Cognitive reloading: Building a new mental model for the next task
-Re-engagement: Getting back into flow
Research from Carnegie Mellon shows that even brief interruptions can increase task completion time by up to 23%. For complex cognitive work like programming, this cost multiplies dramatically.
Here’s another article from CMU discussing the same thing: https://www.sei.cmu.edu/blog/addressing-the-detrimental-effects-of-context-switching-with-devops/
What this study shows is that a person who is faced with an unexpected context switch performs worse on a task than a user who has spent the last 12 questions performing the task the same way.
This exact problem would happen if you replaced AI with a calculator, or made a person swap from using paper to doing mental math. The problem here is context switching, not AI.
The way to ensure that the problem is AI and not the context switch, would be to continue the quest and see if the first group reverts back to baseline after 12 questions. 12 questions is how long the control group had to become acclimated to the task before their last context swap at the start of the test.
Also, of note, this is a paper on arXiv it is not published so it has not gone through a peer-review process which would certainly catch the failure to set a proper control group.
Context switching isn’t just X — it’s Y.
Are we sure this was written by a human?
AI being released was basically an apocalypse for people who use EM dash.
Here’s the most cited, human created (2001), paper on the topic of context switching performance loss: https://www.apa.org/pubs/journals/releases/xhp274763.pdf
Thanks.
And I’m all for em dashes. After all, I started using them after reading enough books. It’s just that particular construct that strikes me as especially LLM-y.
AI was trained on human writing. If it produces a certain tone, then that’s probably a result of the material that was favoured in training it. That construction was common in human writing before it became common in AI too.
What makes it stick out is when AI uses it in contexts where humans normally wouldn’t, but this kind of assertion is common in scientific papers and articles. It would make sense to train an AI on scientific writing, since that tone sounds authoritative and like you have some idea of what you’re talking about.
So I don’t think this is an LLM-construct; it’s an instance of the original style that LLMs copy.
True, but in my experience most people use a comma, not an em dash.
According to a new study by researchers at Carnegie Mellon, MIT, Oxford, and UCLA,
Study should be solid I guess.
participants who were given AI assistants (in this case, a chatbot powered by OpenAI’s GPT-5 model) would have the aid pulled from them without warning during the test
Wow, interesting idea. 👍
where they had their assistant removed, the AI group saw the solve rate fall off a cliff. They had a solve rate about 20% lower
And even worse IMO:
They also had nearly double the skip rate, meaning they simply chose not to solve the questions.
This seems very alarming IMO, because this indicates they lost some of their ability to think constructively on how to actually solve a problem!
I know there have always been some who cried wold every time new technology has become available, like calculators and computers. Even dictionaries were once claimed to be harmful once!
But maybe this time there is a real danger, because AI takes away a lot of the need to actually think creatively and constructively. And that’s an ability we must not lose.The last paragraph of the article is even worse. As it mentions 2 studies that show these effects are also long term!!!
Wow. Now do this with a calculator.
A calculator is not the same problem, it doesn’t reduce our general ability to think critically.
As the study defines critical thinking: yes it does.
The study claims, essentially, relying on a machine that solves a Problem for you, lessens your critical thinking skills.
Their Definition of “critical thinking” is just, at least to me, way Off.
Just because i can conprehend Stuff i read for example, does not show critical thinking. It just shows i can repeat shit i read adequately.
It’s just bad science.
This paper shows that a person who has performed a task 12 times performs better than a person who has never performed the same task.
They also do not properly control for performance loss due to context switching which is a well known contributor to performance loss.
It’s a paper on arXiv, it hasn’t been peer reviewed or published.
No the test is not training, that’s a weird thing to claim. The switch is what is tested, and you disregard that 2 other tests have shown similar results. An actual decline in critical and problem solving thinking.
Here is the paper: https://ai-project-website.github.io/AI-assistance-reduces-persistence/
No the test is not training, that’s a weird thing to claim.
The control group solved 12 questions manually and then the 3 test questions manually. The AI grouped solved 0 questions manually and the 3 test questions manually. One group had 12 more manual math tasks to prepare for the manual math test the other group had 0 and also had to context switch.
The AI-assisted group was dealt a context switch, which results in a pretty severe performance loss. A context switch causes performance loss of around 40% according to this paper, which was peer-reviewed and published and is also the most cited paper on the topic, in the APA: https://www.apa.org/pubs/journals/releases/xhp274763.pdf
The AI-assisted group also did not have 12 questions to adjust to the new context, like the control group did. If they wanted to wipe out the context switching performance loss they should have kept asking questions to see if, after 12 questions, the AI-assisted group had a similar performance.
The switch is what is tested, and you disregard that 2 other tests have shown similar results.
No, they did not switch what was tested. Here is an image from the actual paper.
They were given 12 tasks with one group using AI and another doing mental math and then 3 tasks doing mental math. One group had 12 more tasks worth of preparation than the other.
Nothing, not even the article in theOP, says that they did math and swapped to reading to test.
They did 3 different experiments, in each experiment they gave 12 tasks and then disabled the AI for one group and gave 3 more tasks as a test. At no point did they ask 12 math questions and then finish with 3 reading questions or vice versa. They did 2 experiments using math tasks and 1 experiment using reading comprehension tasks.
So one group had 15 math tasks and one group had 12 ‘how to ask an AI’ tasks and then 3 math questions.
They also did not control for context switching losses, which is a well documented (see the APA paper) effect. The proper control would be to continue asking questions so the AI group also had 12 math tasks before the test.
There’s a reason that this is published on arXiv and not in a peer-reviewed journal. Designing a poor quality experiment doesn’t tell you anything useful even if you do multiple different versions of the same experiment.
This paper demonstrates a lack of a proper control group, specifically a failure to control for context switching performance loss.
The picture you post contradict your claims. The 2 groups are getting the same question, but one has AI assistance, the other has not.
Again you fail to show anything to support your claims.I also wrote text.
If you’re just going to cherry pick a single point and dismiss everything else then we’re done here.
Maybe they’re unable to switch contexts
I fully support skepticism over ai and concerns over its use. But let’s be skeptical about the skeptics. There’s been news in the last week that companies are cutting jobs and blaming ai. I doubt critical thinkers are hanging it up and relying on ai.
Have you ever met an executive? Have you ever met any actual capitalist?
They aren’t particularly smart people. They just have no physical capability for empathy. That is how they can exist. AI is good enough to reduce workload. It is fantastic especially at correcting speech and translating.
You know what worker class needs corrected speech and translating but otherwise can be taught to do most office and entry level jobs? Outsourced workers.
Companies slowed outsourcing customer-facing positions due to backlash from obvious accents and poor cross-cultural training. AI has allowed them full steam ahead. While real time voice masking is a little expensive right now, AI chat agents are good enough to be used while having an outsourced worker listen in, feed ‘correct’ lines to the AI (or simply skip incorrect lines) and actually perform the actions.
GAN ML is also good enough, as it turns out, to figure out how to complete many office tasks with full desktop screen captures.
If you combine these two things, and add a little marketing spin, what you have is a very clear plan to eliminate 50-70% of labor cost in the US – that is the majority of customer service and office administration workers.
Right now it’s AI facing, (statistically) Indian outsourced agent backed solutions. Eventually those outsourced agents – which have the totality of their job recorded, every mouse click, every key press, every single word said to their coworkers and managers, every single blink, all to train AI – will be out of a job too.
Nevermind this ends capitalism, as without a consumer base there will be no companies, but oh wait, techbros and capitalists are pushing for UBI. . . Isn’t that weird they’d be pushing an objectively socialist idea… I wonder if that’s related.
