On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜

Abstract The past 3 years of work in NLP have been characterized by the development and deployment of ever larger language models, especially for English. BERT, its variants, GPT-2/3, and others, most recently Switch-C, have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pretrained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We provide recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.

I can help but I need to ask Grok first

Here’s a paper:

AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking , by Michael Gerlich

The stochastic parrots paper hat a whole section on it, though I believe some of the claims in the paper have been refuted. Either way, it would be a great start for a citation chain to review this.

https://www.truthdig.com/articles/the-ecological-cost-of-ai-is-much-higher-than-you-think/ (from November 2025; the same author Alistair Alexander also has some other good articles there such as this one from last month)

Eryk Salvaggio writes some good articles about AI. A sampling:

Stochastic Flocks and the Critical Problem of ‘Useful’ AI

The Illusion of AGI, or What Language Models Can Do Without Thought

The Computer Science Fetish

They’re not research papers and don’t often deal with environmental impact as the primary topic, but the author does regularly explore the impact of generative AI on society in his writing.

This link contains a lot of references that are good for more than just environmental impact, but many of them are not research papers. But it might turn up some leads for you at at minimum is a good and well thought out article about LLMs (it has about 50 odd references, the environmental ones start at 27 I think)

https://blog.johanneslink.net/2025/11/04/to-gen-or-not-to-gen/