Ok so Samsung, SK Hynix and Micron do not have the capacity to meet demand. Also, what little capacity they do have they are allocating to HBM over DRAM. Based on my limited knowledge HBM can not be easily repurposed for consumer electronics. Translation: main street is cooked for the next 3-4 years.
It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM.
Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
The Radeon VII came out in 2019 as a $700 consumer GPU with an 1TB/s HBM2 memory subsystem which is more than any consumer GPU you can get today, including the high-end ones afaik. At that point in time, there was a whole lineup of AMD GPUs with HBM going down into the midrange.
If they could make this stuff and sell it to regular people a decade ago for very palatable prices, why do they come up with the idea that this is the technology of the gods, unaffordable by mere mortals?
I have been wondering this recently. It was the convention that if you wanted to keep costs down, try to keep the memory bus size down as low as possible. Still remember the awful Radeon 9200 SE - 64bit data bus that strangled an already slow GPU.
Heck, I have a phone with a 16bit memory bus for instance. The high(ish) clock rate only makes up the difference slightly.
But with general prices on all components going up, it might not be such a big factor any more.
HBM migght make sense for higher end products which can free up space for the lower end that will never use the tech.
Yes, it's interesting that HBM was invented by a collaboration between AMD and SK Hynix. It seems, HBM is the way to go for GPUs, anyway.
The GB202 die that's in the GDDR7 based RTX 5090 and RTX 6000 Pro literally needed to be this big to support the 512bit memory bus. It's probably only getting worse with smaller node sizes. (see https://www.youtube.com/watch?v=rCwgAGG2sZQ&t=65s).
BTW: The 1TB/s is matched by RTX4090 and surpassed by the RTX5090 (1,79 TB/s).
5090 is an overpriced outlier. A typical consumer GPU, like RTX 5070, has a 3-times lower memory throughput.
Even a RTX 5080 has a lower memory throughput than a Radeon VII from 2019, 7 years ago, while being much more expensive.
The memory throughput of GPUs per dollar has regressed greatly during the last 5 years, despite the fact that the widths of the GPU memory interfaces have been reduced, in order to decrease the production costs.
RTX 5080 has a 256-bit memory interface, while the much cheaper Radeon VII had an 1024-bit memory interface. RTX 5080 has almost 4-times faster memories than Radeon VII, but it has not used this to increase the memory throughput, but only to reduce the production costs, while simultaneously increasing the product price.
Modern GPUs like RTX 5080 are much faster for the applications that are limited by computational capabilities, mainly because they have more execution units, whose clock frequencies have also increased.
I suppose that most games are limited by computation, so they are indeed much faster on modern GPUs.
However, there are applications that are limited by memory throughput, not by computation, including AI inference and many scientific/technical computing applications.
For such applications, old GPUs with higher memory throughput are still faster.
This is why I am still using an old Radeon VII and a couple of other ancient AMD GPUs with high memory throughput.
Last year I have bought an Intel GPU, which is still slower than my old GPUs, but it at least had very good performance per dollar, competitive with that of the old GPUs, because it was very cheap, while the current AMD and especially NVIDIA GPUs have poor performance per dollar.
5090s are certainly expensive compared to most other GPUs, but not expensive enough to be unobtanium for nearly any professional who could utilize one as part of their job
Vega was a card with decent perf/$ for the consumer, but from a pure technical point of view (perf/mm2, perf/BW, perf/W) it was a major failure. Both Vega (and Fiji before it) showed that excess memory BW alone is not sufficient to win.
The fury X was a beast for a consumer card, sadly limited by having a mere 4gb of (hbm) VRAM, and a non-refillable AIO. But back when games could fit in 4gb, it was incredible.
I'd absolutely buy another hbm consumer GPU if it had at least 8gb (and if I got the vibe/hope AMD will actually support for a couple years...)
FWIW that depends on the stores you're looking at. There are three models from different manufacturers available here in a few shops. The prices are a bit ouchier than what i paid for mine around Christmas 2024 though (i got mine on a sale).
After NVIDIA essentially removed FP64 from consumer GPUs (their 1:64 performance ratio is worse than what you can obtain by software emulation, so it is useless, except for testing programs intended to run on datacenter GPUs), AMD persisted for a few years, but then they also followed NVIDIA.
AMD Hawaii GPUs still had 1:2 FP64:FP32, while the consumer variant of Radeon VII dropped to 1:4. The following AMD consumer GPUs dropped the FP64 performance to levels that are not competitive with CPUs.
Nowadays the only consumer GPUs with decent FP64 performance are the Intel Battlemage GPUs, which have a 1:8 performance ratio, which provides very good performance per dollar.
Only indirectly. They have most of the money, so if they want something that’s in short supply, the price will rise to the point that it becomes unaffordable to everyone else.
Reason number 7,322 why US-style ultracapitalism is self-destructive, anti-social, and dystopian.
To add a more local hurdle as well, the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.
That is, memory capacity is reserved for datacenters yet to be built, but this will do weird things if said datacenter construction is postponed or cancelled altogether.
That guarantee is not as much of a guarantee as stated in the media. You get a guarantee it will be planned at a certain time (as in looked at), not that it will be build. The cost of doing business is taking risks and mitigating them. There is a reason the nuclear plant in Borsele was build: an aluminium smelter. Maybe you should arrange for something similar as a datacenter (no politician will fall on a sword for that but you can try). The (original) power draw is about the same 80-100MW.
It says that in 2025, Netherlands was a net exporter of electricity (~14,000 GWh). My guess: Where they want to build data centers, the grid cannot handle it, but the overall system has more than enough power to build data centers. Do you think that sounds like a resonable guess?
> the Dutch power grid is at capacity and its managing company is now telling companies that planned to build a datacenter that they can't be connected to the grid until 2030, even though said companies already paid for and got guarantees about that connection.
Are the Netherlands a large proportion of global datacenters?
Amsterdam hosts a major internet exchange. It's not a bad place to build a datacenter and there are many. Northern latitude brings free air cooling, but also additional distance to clients. Lots of peers in AMS-IX, but not a lot of oceananic cable landings (one with two paths to the US, but most of the submarine cables land nearby in Europe)
London is about 3X bigger than Amsterdam in terms of capacity. If you look at the core western europe market (so called FLAP-D) which I am rather familiar with London holds 35% of that market and Amsterdam about 12%. I'm not at all sold that the old rule of thumb that this market is about 25% of global capacity has been true since about 2023 because of the AI buildout in NA/ME.
Yes. Amsterdam has one of the largest IXPs (AMS-IX) in Europe and is also one of the largest European markets for Internet Infrastructure services (i.e. hosting, DNS provision, domain name registration, etc.)
Do AI data centers not need internet connectivity anymore?
The value of an IX isn't just in the IX itself, but also in the presence of hundreds of parties for direct peering, and excellent connectivity to the rest of the world.
It makes a lot of sense to build your DC near one - even if you have no intention of actually participating in the IX itself.
> Do AI data centers not need internet connectivity anymore?
They don't need entire IX worth of connectivity. You're sending mostly text back and forth and any media is in far lower volume than even normal far less dense DC would generate, all the major traffic is inside the AI DC.
What the grid looks like in different countries is very different. The Dutch power grid is already almost 50% renewables, which is an inconvenience for adding capacity because that's around where you have to start really dealing with storage in order to add more.
In most other places the percentage is significantly less than that and then you can easily add more of the cheap-but-intermittent stuff because a cloudy day only requires you to make up a 10% shortfall instead of a 50% one, which existing hydro or natural gas plants can handle without new storage when there are more of them to begin with.
I don’t think the source of the electricity is particularly relevant to whether or not you have the transport capacity to add tens of megawatts of demand to the grid. The problem is generally not the supply but whether your local transformers have capacity left.
When you're talking about something that draws megawatts existing transformers are pretty irrelevant because you're going to run high voltage lines directly to the site itself and install new dedicated transformers on site.
What's more common is that they don't have the transmission capacity itself, but that one's pretty easy in this case too, because what that means is that you have an existing transmission line which is already near capacity with generation on one end and customers on the other. So then you just build the data center on the end of the transmission line where the generation is rather than the end where the existing customers are, at which point you can add new generation anywhere you want -- and if you put it near the existing customers you've just freed up transmission capacity because you now have new customers closer to the existing generation and new generation closer to the existing customers.
High-level, I would agree with you. One thing that blows me away: I think I read that Northern Virginia, USA has the highest data center density in the world. Mostly it is due to demand from US gov't, military, and spy agencies (like NSA). How did they do it? In mainstream media, I don't see any news about a stressed power grid in this area. I guess the US gov't carefully coordinated with local power providers to continuously upgrade their power grid? This is a real question. It makes no sense to me. No shilling/trolling here.
> I think I read that Northern Virginia, USA has the highest data center density in the world. Mostly it is due to demand from US gov't, military, and spy agencies (like NSA).
That's where AWS us-east-1 is, i.e. the oldest AWS region where they got started to begin with. Google and Microsoft also have a large presence there. It's not just the US government, it's everybody, and it's not new.
> How did they do it?
Here's the US nuclear plant map, guess where a bunch of them are:
The area around Virginia is also a major coal producer and when this was getting started it was a source of cheap electricity, but coal is quickly being replaced with natural gas via pipelines from the Gulf coast. Their current power mix is ~30% nuclear, ~12% renewables (solar) and almost all the rest natural gas.
Well, they have done pretty well for 20 years of planning. Google tells me that AWS us-east-1 region (Northern Virginia) was started in 2006!
EDIT
The opening paragraph:
> State regulators’ review of the controversial power line proposed to stretch across three rural Maryland counties will extend to at least February 2027, officials announced Thursday, a timeline that prevents developers from meeting the grid operator’s deadline to ensure reliable electricity.
I bet this is pure NIMBYism. Just this phrase alone is a dead giveaway: "controversial power line". LOL: What is controversial about a power line? Hint: They aren't, but NIMBYism exists.
The grid is constantly "stressed" everywhere, because they're designed to be. You don't build five times as much capacity as you need at huge expense for no reason, you design the thing to have only slightly more capacity than you expect to need.
If demand increases you have to build more generation and power lines etc. This is not a problem but for NIMBYs, it's just the logical consequence. If the local population increases and you don't have enough grocery stores you don't say "grocery stores are stressed" and regard it as an insurmountable problem, people just open more of them.
Chef's kiss for this reply. It is perfect. I like how you reframed the argument around grocery stores. It would be ridiculous to write a newspaper article about "grocery stores are stressed" and there is a newly proposed "controversial grocery store". NIMBYists will perform all kinds of mental gymnastics to defend their positions.
I know that you comment is midly off topic, but I am going through the same out of body experience each time I see a major project announces an opening date of 2030 or later.
This just highlights what an utter failure and self-inflicted wound the green policies of Euro countries have been. Europe has already lost the AI race to the U.S. and China.
Renewables is the only realistic path to energy independence. Today's global situation should show the absolute necessity of that, even if you dont give a sh*t about the environment.
Had we done more 10 years ago we would have been better of. The second best time to start is now.
(We used to build it at a fraction of the cost and less than half of the time that we do with our modern fuckups and fuel can come from just about anywhere if need be. It might be a lot more expensive than the stuff kazachstan and still be a fraction of the cost.)
I think ideally we would've done both to press the cost of nuclear down and given the fact that the renewables rollout turned out to be a lot lot more expensive than proponents claimed it would be whilst still tying us up into gass to cover winter.
When reading an article written by Bjorn Lomborg, you should also do the effort to read the cited sources. This is not an ad hominem attack, just an observation. Do it and you will see.
It does not. That is not economically mined. Last big hard coal producer in EU - Poland, has extraction cost x2 or x3 of the mountain top removal mining in US. This sector is shrinking rapidly. Poland coal production came back to ~1915 levels (taking into account current PL territory). This sector would be closed already if not for massive subsidies.
Last year, China's coal use decreased, while China installed 300x more renewables than nuclear. Coal and nuclear aren't cost competitive with renewables, either in a free market or a technocratic top-down economy. Coal and gas still maintain a valid niche of firming intermittency. But that niche is temporary and shrinking.
The free market installs a tiny amount of coal, and a lot of renewable energy. Whether you believe this means "coal is/isn't cost competitive with renewables in a free market" is a debate about word definitions that I'm not terribly interested in.
China, like Brussels, is trying to reduce coal for similar reasons. They don't like the air pollution health hazard (fully believable), and they say they don't like global warming (somewhat believable).
China and Texas are both installing silly amounts of renewables. They install very little new fossil fuels or nuclear. They both maintain cheap electricity prices through abundance.
The problem in the EU is not renewables, it's the same problem that Democratic states in the US face. Regulations and permitting hurdles that block private renewable energy developers.
Their coal generation decreased last year. They're building on the order of 70GW of new coal while they decomission or underutilized more than 70GW of pre-existing coal. Meanwhile they installed 450GW of new renewables energy.
Not relevant to the question of which energy source makes sense to build in the year 2026. But sure China has many coal plants left over from 2003 when renewables was more expensive, nobody would dispute that this is a fact, however irrelevant.
There appears to be zero advantage to having the datacenter actually in your country apart from minor local property tax, in exchange for which it will put up the electricity bills of every single citizen, who already hate how much they're paying.
I am betting the pendulum swings faster to the other side to excess capacity as all the construction lies of Altman fall through with financiers waking up the the fact they can't build the infrasctructure as fast nor make any profits on that infrastructure that will get built.
What if it takes 100 years to get to AGI or we never achieve it? All bets on AGI will just fail over and over again for decades in that case. It seems a bit like saying financiers can't risk not being involved with Faster-than-light travel technology. Yeah, it would change everything if we got it, but betting that we'll get it soon over and over again is probably not going to get you a lot of money.
We've been projecting both FTL and AGI as future possibilities for almost 100 years now. Do LLMs get us a lot closer to AGI? I think they get us a little closer and Moore's "law" making compute faster probably is a much bigger factor, but I think we're still a very very long ways away.
Who has been projecting FTL as a realistic technology ever? FTL is not possible according to the current laws of Physics, while AGI is at least not forbidden by them.
I think this should be something you can answer for yourself by looking at human media and news over the lasy 100 years. I find it hard to belive you haven't ever noticed anyone seriously saying we may possibly have FTL sometime in the future. Incidentially I think I read on HackerNews that Sam Altman has been talking about building Dyson Spheres in the future. I suppose they're not forbidden by the current laws of phyisics either, but I don't know if I would call them a realistic technology.
The IQ of the smartest human, the perfect memory storing and recollection of computers, the fact that it never tires. I don't know if it's AGI but it's already something greater than us.
To show ads you need people to stay on your platform. This is especially true once ads become more intrusive or of lower quality, something the big players seem to gravitate towards to keep revenue up. Google and Meta have ways to lock in users (networking effects, the best search engine available, having your data stored there).
I am not sure if OpenAI has that. Their edge regarding models is small, their strategy currently seems to be "buy ALL the hardware so nobody else can". Users can quite easily switch to other models.
If only there were privacy laws and working antitrust laws. There could also be a law straight up banning chat uses of llms and only allow agentic uses with human review. Would solve a lot of problems for lawmakers worried about AI I think.
The problem in this case seems to have sprung from a lack of collusion. Altman reportedly approached Samsung and SK independently to strike deals for a large chunk of both companies' production. Neither party apparently knew he was negotiating with the other.
If they had actually been communicating or colluding with each other, they would have put the screws to him, making it harder for OpenAI to assert control over the vast majority of the DRAM market.
Failing that, you'd like to think a regulatory agency somewhere would step in to keep a single player from hosing everybody else, but...
> Neither party apparently knew he was negotiating with the other.
I don’t buy it that two of the largest manufacturers of DRAM in the world, from the same country, didn’t know this. Even of you ignore each company’s intelligence teams, that’s also the job of the country’s internal intelligence services, to make sure they know what all companies are doing and then make it so they have the best leverage to gain as much as possible. Both companies would have known “somehow” and played hardball.
How would the country’s internal intelligence services know what’s happening? Yes, by spying. That’s literally their job and they have assets in every critical area in a country. Every institution, every major industry player, they are monitored to a degree by the internal intelligence in every country in the world. There are more nefarious reasons to do this but the ostensible one is that if it’s of strategic importance the country needs to know everything there is to know.
The companies also do a lot of spying themselves, every bit of info could give them an edge.
There are a lot of cyclical businesses that make money every year. It requires careful management. Factories can produce less than full capacity - but you better design for that. you can make money in the worst years without laying anyone off even - but it requires careful attention to details and not over hiring in good times as if they will never end.
Factories working at (significantly) less than full capacity gets a bit harder when you've got one of the most expensive machines on earth working in them, and production lines that'll be out of date in a couple of years
the normal way to do that is by hiring/firing to meet demand, but in the fab business, you have 10s of billions of dollars of capex with relatively little opex. if you're running at <90% capacity, you're losing money.
That is the common way, but there are companies that manage without hiring/firing. (or they hire temp workers). There is a minimum level of capacity you need to run just to keep the lights on, and figuring out how to get that low without impacting your ability to serve the highs is hard. Memory manufactures have not gotten very low, probably for good reasons, but it is something they should work on.
Last year ai folks are all over wallstreet and articles, decrying how hardware folks are a roadblock to new frontiers in AI. They just couldn't print and pack the new chips faster.
but if you don't collude during times of feast you will have famine, and during times of famine you will have famine, in an economy based on feast/famine you must sometimes feast or die.
All of the capital intensive businesses face this issue. Chemicals, Shipping, Semiconductors etc.
You get market signals that the demand is there, you acquire the necessary capital, you spend 5 years to build capacity, but guess what, 5 other market players did the same thing. So now you are doomed, because the market is flooded and you have low cash flow since you need to drop prices to compete for pennies.
Now you cannot find capital, you don't invest, but guess what, neither your competitors did. So now the demand is higher than the supply. Your price per unit sold skyrocketed, but you don't have enough capacity!
Forecasting demand 5 years into the future is intrinsically highly unreliable. It doesn’t matter if it is capitalism or a command economy. The bet is always going to be risky and someone will have to pay for that risk.
At least with capitalism you have many different people with different perspectives on the risk making independent bets. That mitigates the more extreme negative outcomes.
To add to that, investors who do make the bet get punished for over-building, which is better than tax payers paying for it. And before someone says it, big corps do get bailed out by gov't, but that's definitely goes against capitalist ideas.
Also it is government job to regulate. Monopolies should be busted, and behaviour like that should be culled. But US govt is full on AI to mask them cratering their own economy.
Is the DRAM industry really capitalist? Focusing on just the Korean parties, it functions like a command economy. I would say the same about most high end semi-conductor manufacturing, TSMC, Intel, ASML are being commanded and driven by nation-state level decision making. Right now the command is to focus on high wattage centralized AI systems at the expense of everything else.
No one at high levels is capitalist, in ideology or action. An ideological capitalist would be in favor of competition, but these people disdain it and collude regularly. The only 'capitalist' actions they take are by accident, the real goal is as much power/money as possible as fast as possible.
We don't even expect companies to plan long-term anymore, it's just moving wealth as fast as possible.
That isn't really a change, very few people could ever have been said to be ideological capitalists. (capitalist is not a word with a hard definition, but I'm considering it a different thing than the more modern pure libertarian zero-regulation ideology)
I think the common use of the term capitalist is as a participant in capitalism.
This is distinct from someone who is a proponent of capitalism as a system, which appears to be the way you are using capitalist. For which I don't blame you.
Liberalism (in the traditional economic sens) likes competition. Capitalism is a mode of production, and capitalists notoriously don’t like competition when they are the incumbents
Because that does not happen exactly as you say for all players. The demand signals will be processed and long-term risk is balanced against short-term gain in a distributed fashion, so not everyone will do the same.
It's more optimal than planned economies until we have AI planned economies with realtime feedback, I guess.
Consumers get cheap goods during oversupply and most inefficient companies get elliminated during bust while consolidation leads to economies of scale.
Why is the opposite of capitalist markets automatically assumed to be a command economy? Co-op style businesses aren't really capitalist orientated but are also not reliant on government action.
That's not what capitalists claim. Capitalists claim communism is responsible for tens, if not hundreds, of millions of death due to famine. And overall miserable ways of life, as if humans were termites deprived of any individualism and any freedom.
Capitalists claim that France importing 500 000 people from the third world each year, out of which only 10% are ever going to work (and these are official numbers) and yet offering them all a safety net is unsustainable. And that that socialism is only going to lead to one thing: running out of taxpayers' money.
Capitalists don't claim they have the best system: what they claim is that they haven't seen a less worse one.
I don't know who these "capitalists" are, but generally speaking business regularizes as entrepreneurs learn to read new signals after unexpected market shifts.
It is unlike a socialist system in that there are signals to read in the first place. What, do socialists claim that failure is optimal when you can't even tell if you've failed?
The sheer fucking blazing ignorance of this comment
> Capitalists claim that this is optimal.
Compared to starving under communism coz someone at top got the number wrong, yes. And it only really happens when there are massive, unpredictable market movements and governments not doing their job. Govt should look at the whole thing and just say "no", blame them.
No market system self regulates well enough, and it's government job to file down the edge cases like this. But the revolution happened in country which has two utterly incompetent parties, both in pockets of billionaires, fighting for power, and the clowns from one that won last battle use AI to smokescreen the economic growth their actions cratered
Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh. I do hope they end up holding the bag once again, cause after covid and the cartel thing they don't seem to ever learn their lesson on how to have the tiniest amount of integrity.
> Memory makers did get themselves into this situation by selling all wafers for empty promises and alienating everyone but OpenAI tbh.
Wasn't the problem here that OpenAI was negotiating with Samsung and SK Hynix at the same time without the other one knowing about it? People only realized the implications when they announced both deals at once.
Permanent public ownership of (very large stakes in) these companies doesn't seem like such a bad idea anymore, does it? It's what we used to have for most of the 20th century at least in Europe.
That's only sound is it not. If you take away the hype - nothing critical actually depends on LLMs. You can remove them all today, and nothing bad would happen.
> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
I hope they do, they did not have to agree to sell so much RAM to one customer. They’ve been caught colluding and price fixing more than once, I hope they take it in the shorts and new competitors arise or they go bankrupt and new management takes over the existing plants.
Don’t put all your eggs in the one basket is how the old saying goes.
> Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?
We aren't. The remaining memory manufacturers fear getting caught in a "pork cycle" yet again - that is why there's only the three large ones left anyway.
If they don't expand capacity much, the only negative consequences I foresee happening for them is that they might lose spending discipline, and that systems will be set up to make do with a little less memory. Apart from that, it's just very high profits followed by more or less regular profits.
They could wind up losing all their business to China though.
China has memory makers who are creeping up through the stages of production maturity, and once they hit then there's no going back.
If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume - that domestic market is huge so they have scale, and the gaming market isn't going to care because they get anything at the moment, which is all you'll need for enterprise to say "are we really afraid of memory in this business?"
Good point, it's a risk but so far the Chinese competition isn't up to par and it's unclear whether they'll be able to exploit the current window of opportunity.
You think this window is short? We've been dealing with this for years and years, and to me it seems more like incumbent manufacturers are too comfortable milking cash cows.
That's only for normal RAM. I'm talking about broader aspect: HBM shortage and high prices have been around for longer, and Chinese manufacturers are also climbing up and expanding there.
> If the existing makers can't meet supply such that Chinese exports get their foot in the door, they may find they never get ahead again due to volume
The answer to that is government regulation. Ban anything Chinese or slap it with tariffs. That is what tariffs are intended for - not for the BS the current administration has done.
Good luck actually pulling that off, it'll just kneecap US/Western industry, just as the current tariff and war madness has helped drag the US economy into the gutter.
Surely this can be solved with financial engineering. The memory makers build more capacity, but they finance it with something like floating-rate notes linked to an index of memory prices, or even catastrophe bonds or AT1s. Or more crudely, set up special purpose vehicles to build the extra capacity, and issue convertible bonds from those; if the memory market collapses, investors don't get paid, but they do get a memory factory.
But wouldn’t you rather hbm prices come down first ? Memory makers will be fine. There is practically infinite demand.
Unless you get china style rationing of compute per person world wide.
The real issue is everyone wanting to upgrade to hbm, ddr5, and nvme5 at the same time.
HBM is just normal DDR RAM that's been packaged with (much) wider-than-usual data buses. That's where the high bandwidth comes from, not from high clock rates or any other innovation or improvement in core specifications.
Anything where there's _at worst_ solder and traces between the compute and the memory. That's why you see it on GPUs (and Apple hardware). DRAMs advantage is modularity.
I think the datacenters will simple be repurposed for hardware as a service, specially cloud gaming, sense it will be to expensive to buy a new computer anyway.
Not a rec, but just my source: Atrioc (streamer, YouTuber) is good at gathering all the facts for the rest of us. There's many other things in play, like the Strait of Hormuz (helium, bromine). Ultimately it works out that the shortage, and shortage profits, will continue; the chip makers are probably going to continue to see record profits (as Samsung has).
The specific mix of factors could change at any time, but the supply chain is relatively inelastic, it will take some time to show up on price labels.
It's not going to last until 2028, it'll last until 'min(AI_bubble_burst, 2028)', which I expect will be a lot smaller than just '2028'. So the real question is, how long will it take to retool for non-HBM, and will there be a fire sale as they scramble to recover?
Which also explains why production is falling behind demand, companies aren't going to sink billions into creating product for a market that could dry up overnight.
I think I’m missing something. Financially, what bag would the memory makers be holding here? I don’t think I’m well informed regarding how these deals were structured.
Memory makers make capital investements (build different factories, convert physical production lines, etc.) to meet orders that have been place for the next ~5 years.
OpenAI (or whoever) crashes and can't pay for the order leaving the memory makers in a tough spot.
The amount of money flowing both from the AI bubble and from quite literally scalping both the server and consumer market... They gambled on the opportunity and if they fail - it's their problem.
And how is that a problem and more importantly how's that a problem of Average Joe?
Capitalists did their gamble things. If they fail in that gamble what forbids them to sell the regular RAM they made for AI bubbleists to the regular consumers? Besides HBM it's just the regular chips which are exactly the same for the consumer/server market, why it would be any different?
I'm under the impression OAI wrote Letters of Intent, and are not actually on the hook for the RAM they requested.
The others...they did not. Memory makers won't be holding the bag because Apple/Google/Samsung are contractually obligated to purchase, after the panic OAI caused.
The market is already stagnated. Even if OpenAI doesn’t buy what they reserved other players will do so. SK Hynix CEO said there is a 20% gap between supply and demand per year. And that doesn’t account the shock effect that will take place the moment prices normalize and everyone and their dog will go out and start buying inventory to avoid the next crisis. I for one would certainly buy more than I currently need just in case.
> OpenAI’s rapid growth, fueled by the success of ChatGPT and other AI products, led to a landmark agreement in October to purchase 900,000 DRAM wafers per month from Samsung and SK Hynix—amounting to roughly 40% of global supply. This surge in demand, coupled with limited manufacturing capacity, sent prices for memory kits skyrocketing. [0]
They ordered 40% of the global RAM production for 2025/26. It was a non-binding agreement that either side could easily withdraw from but they're essentially trying to buy about half of all the RAM.
If I booked half a hotel's rooms then suddenly said "yeah never mind. Half my friends cancelled and we're not staying", basically any hotel would be coming at me for my money because there's no way they can fill their rooms now and they're losing revenue. But OpenAI can really get the whole world to pivot towards it then say "cool but we don't need your product anymore" and RAM makers are just going to let it go.
Whoever decided that was a good idea needs to be fired and publicly shamed.
Well if that hotel was then able to sell the other half of hotel rooms for 10x the old price. Then the hotel might actually be happy as they can now charge 10x for the other half or slowly lower prices back down over years.
Do the memory makers not have a contract in place for an order this large? I assume that they aren't going to take "trust us bro" as good enough for several million dollars in orders, and even if there is a way to cancel the order it won't be free. I would assume so at least, but i would like if anyone knew for certain.
I was interning at a company that made networking gear that was put out of business when their largest customer canceled an order within a week of the delivery date.
The customer ran out of money. In terms of where you are in line of debtors when you haven't even delivered the product to a customer, it's so far back as to be assured you won't get your money.
If the memory makers got a deposit from OpenAI as part of this deal, that is likely to be the only money they will get for any undelivered memory, particularly if OpenAI runs out of capital.
It's a business with huge up-front capital expenses and typically very low margins. Supply is scaling up slowly because it's hard, and if you overshoot, you go out of business.
Nobody is "allowing" this. It's a natural property of being both advanced technology and a commodity at the same time.
The strange deals on the entire future output are what was allowed. Try to do the same thing with onions and the government understands you are a criminal.
That is quite the amusing read but it seems like a poorly constructed law. It wasn't futures themselves that were the problem there. The duo engaged in blatant market manipulation and severely disrupted part of the food supply in the process.
Cornering the market with the intent to flip the goods is not quite the same as cornering the market because you actually want the goods and intend to use them yourself.
And it just so happens that many people will now have to use OpenAI’s products because they can’t get enough RAM to run a local LLM. What a coincidence.
It has the makings of a natural monopoly, except its compounded by RAM cartels colluding to shut out the last of the competitors.
Recently they had a second price fixing lawsuit thrown out (in the US).
Now with the state of things I'm sure another lawsuit will arrive and be thrown out because the government will do anything to keep the AI bubble rolling and a price fixing suit will be a threat to national security, somehow. Obviously thats speculative and opinion but to be clear, people are allowing it. There are and more so were things that could be done.
Allowed? We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.
It started with raegan, and even parties on the “left” in the west believe in it with very few exceptions.
> We live in a neoliberal world where corporate monopolies / oligopolies aren’t even remotely regulated. If you try to do even the gentlest regulation of companies people scream about communism and totalitarianism. Unless the regulation serves the monopolies by making it harder to enter the market.
The thing that enables this is pretty obvious. The population is divided into two camps, the first of which holds the heuristic that regulations are "communism and totalitarianism" and this camp is used to prevent e.g. antitrust rules/enforcement. The second camp holds the heuristic that companies need to be aggressively "regulated" and this camp is used to create/sustain rules making it harder to enter the market.
The problem is that ordinary people don't have the resources to dive into the details of any given proposal but the companies do. So what we need is a simple heuristic for ordinary people to distinguish them: Make the majority of "regulations" apply only to companies with more than 20% market share. No one is allowed to dump industrial waste in the river but only dominant companies have bureaucratic reporting requirements etc. Allow private lawsuits against dominant companies for certain offenses but only government-initiated prosecutions against smaller ones, the latter preventing incumbents from miring new challengers in litigation and requiring proof beyond a reasonable doubt.
This even makes logical sense, because most of the rules are attempts to mitigate an uncompetitive market, so applying them to new entrants or markets with >5 competitors is more likely to be deleterious, i.e. drive further consolidation. Whereas if the market is already consolidated then the thicket of rules constrains the incumbents from abusing their dominance in the uncompetitive market while encouraging new entrants who are below the threshold.
Arguably a more efficient approach might just be to have a tax that adds on to corporate tax incrementally for every % of market share a company has above say 7-8%. Then dominant companies are incentivised to re-invest in improving their efficiencies rather than just buying/squeezing out competitors. A more evenly spread market would then, as a result, be against regulations that make smaller market participants less competitive, as they'd all be in relatively less table positions.
> Arguably a more efficient approach might just be to have a tax that adds on to corporate tax incrementally for every % of market share a company has above say 7-8%.
How is this more efficient? You'd still be applying all of the inefficient regulatory rules intended to mitigate a lack of competition to the smaller companies trying to sustain a competitive market, and those rules are much more deleterious for smaller entities than higher tax rates.
If you have $100M in fixed regulatory overhead for a larger company with $10B in profit, it's only equivalent to a 1% tax. The same $100M for a smaller company with $50M in profit is a 200% tax. There is no tax rate you can impose on the larger company to make up for it because the overhead destroys the smaller company regardless of what you do to the larger one.
First, on the surface, this sounds like a terrible idea. Almost all ideas that I see on HN about economics fail with even the tiniest amount of common sense.
As a counterpoint: Look at very high value goods, like jet engines and MRI machines. I went for an MRI the other day and wondered to myself (then asked an LLM) what the international MRI market looks like. They are vanishingly small number of manufacturers and are usually dominated by a few international players. How are you going to apply this tax to non-domiciled (international) companies? Also, companies like General Electric, Mitsubishi Heavy, and Seimens are enormous and incredibly diverse. This idea falls apart quickly.
Maybe it’s time for a refresher on what neoliberal means? It’s not simply “new liberalism”. Reaganomics was the start of neoliberalism in the US, tho of course it shifted and developed its character further over time into the monster that drives 99% of our problems today
This will result in demand destruction which will starve the enterprise which will starve the hyperscaler. theres no situation where people not being able to afford hardware for 4 years results in the bubble not popping
I'd expect unaffordable hardware to drive demand for thin clients connected to cloud services which is something that had already been happening gradually prior to this.
They won't be, prices are high because they are refusing to build capacity for demand that may evaporated by the time they are done. They are holding back and building only enough so when the bubble pops they will be fine.
You can't build capacity overnight, and even with that in mind, it's hard to say if it is sensible to increase capacity now that we are in an AI bubble. For all we know, the bubble might burst.
So the ML hate is weaponized in the form of memory demand collapse FUD, and the public at large has to pay through their nose for it... thanks party poopers!
I don't think its from the ML collapse FUD, its most likely from the multiple time's in the past when they overbuilt and it resulted in a memory oversupply and price collapses. The 1985–1988, 1993–1994, 1998–2002 and the post pandemic oversupply. These were all cases where shortages followed by over corrections caused oversupply, financial losses due to low prices and fewer surviving companies. I think they're taking their time and are cautiously adding more capacity in such a way that prices won't end up collapsing again. Regardless, the result is still that we the consumers have to pay more.
At this point the remaining memory companies are… the ones that didn’t die during an over-supply collapse, right? I guess there’s been a strong evolutionary pressure against giving consumers what we want, haha.
If they gradually increase production capacity then prices stay high for 10+ years (or for as long as it takes for demand to crash) because a gradual increase in production takes that long for them to add enough capacity for current demand.
If they add enough capacity to meet current demand quickly then if demand crashes they still have billions of dollars in loans used to build capacity for demand that no longer exists and then they go bankrupt.
The biggest problem is predicting future demand, because it often declines quickly rather than gradually.
do we have evidence of RAM manufacturers going bankrupt? do we have evidence that the increased capacities after the mentioned past shortages went unused or were operated at a loss?
Machines take up space in buildings (factories); both of which are discrete rather than continuous functions. If your factory is already full of memory-making machines, and want to add one more, it will cost you billions and many months to build another factory.
If you suppose you have cracked the smooth-ramping problem, perhaps you should throw your hat in the ring and soak up all the pent-up demand that SK Hynix, Samsung and Micron are neglecting.
Think of the factory problem from physics first principals instead, as Elon would say. Musk says he will outcompete earth fabs by building them on the moon in just a few years, deploy radiation harden versions of the chips into space, and beat out TCO vs doing this on earth.
If he can do all that that fast, the RAM makers should be able to at least 1000X their fab capacity on earth in one year. One year for scaling up existing tech is an eternity compared to Elon's timeframe for moon-fabs given the relative complexity of the challenge.
The most well known thing about Musk besides being an asshat, is that his timelines are almost always imaginary. He is not building fabs on the Moon in "just a few years".
He doesn't have to build them to sell them. FSD has been sold for almost a decade, more than half the depreciation time of the cars. With first principles sales techniques it's possible X could sell these moon chips now and deliver them made on earth 6 or 7 Moore's law cycles later by a different company, like with the solar roof or hydraulic brick machine from alibaba, way cheaper.
But how is that at all related to the DRAM manufacturers' short term production limitations? Unless I'm being dumb and you're saying that these manufacturers could also just kick the can down the road for a few fab generations?
I would expect that OpenAI gets as much money as they ask for for the next 10 years.
There’s virtually infinite capital: if needed, more can be reallocated from the federal government (funded with debt), from public companies (funded with people’s retirement funds), from people’s pockets via wealth redistribution upwards, from offshore investment.
They will be allowed to strangle any part of the supply chain they want.
China already has a well developed DRAM industry, as DRAM is somewhat easier than logic, and can tolerate a much higher defect rate. The industry will figure this out.
Another point is I often see the money argument - like country X has more money, so they can afford to do more and better R&D, make more stuff.
This stuff comes out of factories, that need to be built, the machinery procured, engineers trained and hired.
I think the article has a giant blind spot as far as China is concerned , considering they have already a mature enough memory ecosystem via YMTC that Apple was considering sourcing from them. As well as continued expansion in the DRAM and HBM Fabs [1].
It feels like the memory cartel once again trying to incentivise their various govt to cough up some more tax breaks/funding to cushion the AI buildout bet that they made and the bubble seeming about to pop.
In any case if they leave the consumer market underserved it should be no surprise if before that 2030 prediction we are all on cheaper YMTC memory modules.
I think you're massively overestimating how much money is really accessible here. The parent comment's right that all of the easily available VC & private equity investment is basically used up. OpenAI was struggling to sell $600M of private equity, the big multi-billion dollar investment packages had lots of conditions and non-cash in it.
> more can be reallocated from the federal government (funded with debt)
While this is the most reliable funding, it's still not very accessible. OpenAI is a money pit, and their demands are growing quickly. The US government has started a bunch of very expensive spending. If OpenAI were to require yearly bundles of it's recent "$120B" deal, that's 6% of the US' discretionary budget. 12.5% of the non-military discretionary budget. (And the military is going to ask for a lot more money this year) Even the idea of just issuing more debt is dubious because they're going to want to do that to pay for the wars that are rapidly spiralling out of control.
None of this is saying that the US government can't or wouldn't pay for it, but it's non trivial and it's unclear how much Altman can threaten the US government "give me a trillion dollars or the economy explodes" without consequences.
Further deficit-spending isn't without it's risks for the US government either. Interests rates are already creeping up, and a careless explosion of deficit may well trigger a debt crisis.
> from public companies (funded with people’s retirement funds)
This would be at great cost. OpenAI would need to open up about it's financial performance to go public itself. With it's CFO being put on what is effectively Administrative Leave for pushing against going public, we can assume the financials are so catastrophic an IPO might bomb and take the company down with it. Nobody's going to be investing privately in a company that has no public takers.
Getting money through other companies is also running into limits. Big Tech has deep pockets but they've already started slowing down, switching to debt to finance AI investment, and similarly are increasingly pressured by their own shareholders to show results.
> from people’s pockets via wealth redistribution upwards
The practical mechanism of this is "AI companies raise their prices". That might also just crash the bubble if demand evaporates. For all the hype, the productivity benefit hasn't really shown up in economy-wide aggregates. The moment AI becomes "expensive", all the casual users will drop it. And the non-casual users are likely to follow. The idea of "AI tokens" as a job perk is cute, but exceedingly few are going to accept lower salary in order to use AI at their job.
There's simply not much money to take out of people's pockets these days, with how high cost of living has gotten.
> from offshore investment.
This is a pretty good source of money. The wealthy Arabian oil states have very deep slush funds, extensively investing in AI to get ties to US businesses and in the hope of diversifying their resource economies.
The "no food in other countries" is because of failed/corrupt governments, not because people use AI to generate cat pictures in the West. The economy is not a "fixed pie" that needs to be allocated among people of the world.
Just look at Cuba, which could be a very rich country and one of the prime tourist destinations of the world.
It doesn't stop there though. OpenAI is currently mired in a capital crunch. Their last round just about sucked all the dry powder out of the private markets. Folks are now starting to ask difficult questions about their burn rate and revenue. It is increasingly looking like they might not commit to the purchase order they made which kick-started this whole panic over RAM.
Soo ... how sure are we that the memory makers themselves are not going to be the ones holding the bag?