There Is No Cloud, Just Somebody Else’s Broken & Bullshitting Weather Model.
At least I won’t know about the typhoon when it comes for me, saving me so much stress! Thank you President Trump!
Now americans can be served climate crisis denial with matching weather forecasts.
Okay, a lot to unpack here. I work on atmospheric model development. Primarily upper atmosphere models, but have experience with forecasting models as well. Inherently, there is nothing wrong with moving to the cloud for hpc resources. However I do take an issue with the following statement:
“Cloud-based high-performance computing will accelerate the transition of research into operations by eliminating traditional bottlenecks of on-premise systems,” said NOAA Administrator Neil Jacobs in a statement.
The bottleneck in forecasting is NOT hardware but unknown and unresolved physical processes. Even at high resolution, forecasting models cannot capture physics and dynamics at spatial scales smaller than the grid of the model. We can make assumptions and use things like machine learning to improve subgrid processes but we cannot fully resolve them without an infinitely small grid, which is impossible. So moving to the cloud will not improve forecasting or the transition of RTO.
The second issue with this guy is he is a Trump nut hugger. See the following:
Sharpiegate https://en.wikipedia.org/wiki/Neil_Jacobs
I completely agree with everything you’ve said, but I think Neil meant that on-premise systems have bottlenecks that cloud resources do not. Specifically, on-premises systems have a hard upper limit on available compute resources, whereas in the cloud you can just purchase more compute resources as you need them.
How in the world does a remote computer provide faster results than the one sitting in your building?
Ok hey we control the government now and we fired all your staff who run the computers and now you have to use our services!
Also, your boss loves us and HATES you, like seriously wow we could so easily tip this thing over if we started making a stink about the climate scientists being difficult…
Pats the dead corpse of NOAA on the back in a way that is meant to intimidate it.
Truth
The simple answer is cloud resources can scale to any number of cpus and gpus whereas on prem is static
The remote one is faster.
The time to display the result is far lower than the time to calculate it.
faster at rationalizing austerity for life-saving and cornerstone government institutions?
Faster at computing things.
“Under the contract, NOAA can also avail itself of Google’s DeepMind set of AI tools to help build out its first AI-driven weather forecasting system, the AI Global Forecast System, which promises to offer accurate weather forecasts using 99.7% fewer computer cycles and take minutes, rather than hours, to produce a forecast.”
If there’s one thing I know about large tech companies and promises, they always keep them. Thankfully the U.S. doesn’t have a bunch of sociopaths running the government who don’t want to acknowledge climate change because oil and coal industries are giving them bribes. This will only end well.
Please note these “AI” weather models have nothing to do with “AI” LLMs. Weather models are actually exactly where you want to use machine learning models. It’s huge amounts of data and we know there’s patterns in there, but they are too chaotic for us to easily get from the data. Large machine learning models can parse all of the data and use the model they create to feed in current data and get predictions on future data. There’s all sorts of neat maths you can get involved and with the very powerful hardware available these days, plus the huge improvements in software to drive that hardware, means weather prediction is getting better fast.
It’s kinda sad all machine learning gets the “AI” label these days. And sure these models make mistakes, sometimes big ones. That’s why often multiple models are used and people are involved to see the most likely one. And the truth is these models are doing better than the old models did. It’s the weather after all, it’s a chaotic system with a huge number of variables, it’s basically impossible to get it right all of the time. But at least where I live, it’s been getting very accurate, especially in the 24-48h range. The 5 day prediction is also getting better, but still gets it very wrong sometimes.
I don’t trust computer people with keeping rational boundaries with AI-type pattern matching/corner cutting tools, there is little evidence it working out will be a likely longterm outcome based on their past behavior, if they weren’t trained as Scientists and not just Computer Experts they shouldn’t be let within 1000 kilometers of NOAA’s Climate Models and I am extremely skeptical there are large scale gains to be had here at least without actual practical efficient and functional Quantum Computers which I feel ridiculous even saying…
Pattern matching always looks more efficient than Understanding because it is… at first.
I really don’t care if there are efficiencies to be had here with insert new marketing term for machine learning, the trust has been forever carpet bombed.
AI is what happens when Computer Experts convince themselves they are Scientists.
The difference is really quite simple, a Scientist is motivated by curiosity whereas a Computer Expert is motivated by ego to prove problems are perfect nails for their hammers… which is a lot more like a marketing department if you stop and think about it.
Computer science is not a monolith and it’s frankly concerning that you would loop an entire field into “AI bros”.
I think you just want a boogeyman to hate tbh.
looks at computer science
looks in mirror
looks at computer science
yes I am the concerning one
Yeah man, exactly. There’s no nuance to it at all. Everyone in CS is the same. Public scientists, salaried code monkies, FOSS contributors, game developers, robotics engineers, crypto bros. All of us, slathering at the mouse to shove AI and ML into everything just to appease Sam Altman.
It can’t possibly be that a small subset of extremely well funded losers is abusing a subsection of the field that’s recently had major developments without enough of a regulatory oversight.
All chemists are responsible for monsanto All biologists are responsible for plum island All nuclear engineers are responsible for Hiroshima and Nagasaki.
Stuff that shit back where it came from.
Have some damn shame for what computer science has done for human rights abuse and economic fraud in the last handful of years alone.
Yeah the problem with learning is that you might learn something you can use for Evillll.
mayyybe instead you should focus your energy on public agencies and politicians failure to produce guardrails in the form of regulations or policies?
Don’t condescend me by saying that I am unable to learn about things that hurt people, my point is rather what computer people have done makes machine guns look obsolete in terms of pointless violent capacity… and yet y’all still have a hard time accepting the blood on your hands came from computers being used to stab humanity over and over again.
Example A1: Microsoft’s Azure was and likely still is the keystone CRITICAL piece to the Palestinian Genocide being practically feasible.
I know the machines I spend time obsessing over are at the end of the day brutal killing machines, the question is do you?
You are right when it comes to LLMs. However these weather AI models are created by climate scientists and meteorologists, along with a whole bunch of maths nerds. I have a friend that works on these kinds of systems for our local meteorology service. He’s a 100% maths nerd.
Calling these things AI is very unfortunate. They’ve been using these kinds of machine learning algorithms for decades, long before the time of ChatGPT. The label AI has been applied to it from a computer science point of view, but these days people associate AI with LLMs. I can’t stress enough how these models are NOT LLMs, they aren’t used like one would use LLMs and it isn’t like they prompt the model “Give me the weather pls kthxbye”. It gets fed HUGE amounts of data and outputs HUGE amounts of data, all of that data gets used by actual scientists to create the forecasts.
And the end of the day your distinction here is irrelevant, that is what is so sad about it.
All machine learning techniques are expressions that high quality, well structured datasets can be distilled/butchered into brute force solutions given enough violent churning. A computer person becomes obsessed with the Ouija Board-esque nature of this in a casino bliss whereas a scientist steps back and asks “how is this an improvement on the work done to create the well structured dataset used to train this “tool”? No this is a lossy, low quality data compression operation done for no good reason…”
Outside of relational programming it is bullshitting/pattern matching all the way down with this crap and sure sometimes it is useful in the details of complex models but whopee great but a revolution these technologies are not.
edit Ok fine the Functional Programmers in the back are chill too but that is a whole different thing and they are screaming about the same shit hopefully…
