Summary
- AI demand is driving up RAM costs, making consumer sticks pricier.
- Altman implies underused PC RAM should be in cloud data centers for higher utilization in an interview.
- His remarks suggest a preference for cloud over consumer RAM, which, if true, points to a worrying trend.
If you've been RAM shopping in the last year or so, you've probably noticed memory costs quite a bit more than they used to. Part of the reason is the increase in RAM usage for AI, as companies buy large batches of sticks to meet their needs. Now, Sam Altman has gone on record saying it's "quite terrible" that some of the limited RAM in the world is in people's PCs.
People using the RAM in their PCs 5% of the time is "quite terrible," says OpenAI's Sam Altman
It was from a "global allocation perspective," but it's still a worrying one
Over in an interview on the Cheeky Pint (via IGN), Sam Altman discussed the current state of RAM. In the interview, he brings up that people want to run open-source AI on their own hardware. However, according to Altman, that's not a great thing:
From a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer, and being used maybe 5% of the time. Like, you want maximum utilization in the cloud if you're trying to deal with this very supply constrained environment. And somehow many people's intuition is the opposite of that.
It seems his point was that, in a shortage, he would prefer RAM in data centers, where it'll be put to work for hours a day, rather than on someone's computer, where it'll be used only a fraction of the time. And from a "global allocation perspective," I can see what he means.
But overall, I feel his statement has partially shown his hand and disposition toward consumers, particularly the last sentence: "And somehow many people's intuition is the opposite of that." I got the feeling that he thinks RAM sticks in his data centers are worth more than the RAM in people's PCs, whether they run AI off them or not. As if people should let OpenAI have the lion's share of RAM so it can run more powerful AI models in the cloud. And that really comes off poorly in my eyes.
Regardless, this is inference and speculation from a single interview, and it's not indicative of OpenAI's overall stance on the RAM shortage. However, I'd be lying if I said it didn't worry me at least a little bit.
