Rendered at 17:59:11 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
ndom91 5 hours ago [-]
Shameless self promotion: I put together an AI infra built-out financials data-vis project/page recently. It includes graphs and tracks updates to stuff like GPU depreciation schedules like this.
I love the presentation of this. Phenomenal work, thank you.
Bears are gonna eat.
gavmor 4 hours ago [-]
Oops, I read it as bullish. 98.6% utilization? Seems like demand for more capacity.
analognoise 4 hours ago [-]
I agree, and it can’t be right - is this based on “projected utilization” or something?
My thinking is there wouldn’t be downward pricing pressure on data center compute if utilization was actually this high - it would mean providers could jack up prices and walk with larger margins rather than burn money for acquisition, right?
Plus the market wouldn’t increase the price of borrowed money if utilization was that high; they’d see the same signal. But the cost to borrow money for datacenters has roughly doubled year on year (might just be for riskier ventures? Buildings? Is this aggregate?) - those two signals wouldn’t disagree, it’s nonsensical (I mean, right?)
Edit: it’s just “the buildings”, and it makes sense: maximal utility comes from installing compute in a building already setup to house compute (cooling, infra, power). I actually take this as a negative signal: we’re seeing downward pricing pressure already, AND new projects aren’t fully booked, AND money is now twice as expensive. Somebody’s going to lose their shirt unless OpenAI’s Omnissiah comes online and converts us all.
linuxftw 4 hours ago [-]
Thanks for sharing. Counterpoint: H100's MSRP for around $30k. For 6 months at $12/hr, that's $51,840 revenue per card. So they're hitting break even in 3 months, everything after that is profit (minus electricity, etc), and the cards are still selling for around MSRP used.
The real question IMO is building capex. Those depreciate over a longer timeline, but if demand were to drop in 5 years and we have an excess of datacenter space, that's going to hurt. We're approaching the physical limits of what's possible with silicon and fiber optics. Unless there are new architectures or break throughs in algorithms, the only way to make AI more productive today is to build larger clusters, or much longer training/RL runs.
pcthrowaway 30 minutes ago [-]
One thing about the GPUs that I think is unmentioned, is that the LLM models are also becoming more efficient, right? So you can get more tokens or more "work" out of the same GPU as the technology improves
paxy 4 hours ago [-]
- GPUs last a lot longer than 3 years.
- The factories don't stop working after 3 years. They can keep making GPUs.
- The AI boom isn't just about GPUs. There have been massive investments in CPUs, RAM, networking and routing components, physical data centers, power plants, cooling, and even fiber (the thing you yourself claim lasts decades). None of this is going to disappear in a couple years.
whynotminot 4 hours ago [-]
I recommend looking at any data regarding A100 rents (6 year old GPU) and then save 50 minutes of your time.
samuelknight 4 hours ago [-]
Same with the consumer variants. RTX 3090 launched at $1499 and used ones on ebay are over $1500. GPUs large enough and new enough to do inference are not losing value.
Volta generation prices are collapsed, so GPU depreciation is currently around 8 or 9 years
molyss 4 hours ago [-]
What are we looking for? What’s your time-saving insight?
4 hours ago [-]
paxy 4 hours ago [-]
The fact that the claim "GPUs last three years" is idiotic.
jubilanti 4 hours ago [-]
[flagged]
nyeah 4 hours ago [-]
Misinformation is allowed. Incivility is not. See site guidelines.
ActionHank 4 hours ago [-]
Thats the weird thing, scarcity drives value up so long as there is scarcity.
Once there is enough, or even a glut, it's over.
whynotminot 3 hours ago [-]
And I think the counterpoint here would be: Will there ever be enough intelligence or will we just want and use as much as we can possibly build?
Is intelligence like for example food where if I get my daily calories I’m good? Or is there no limit to the demand for intelligence. If there’s more available, will we just keep using more? Forever? It can’t be limitless right?
But where’s the limit because right now we’re consuming every GPU hour we can.
ActionHank 2 hours ago [-]
I suspect that once the hype finally settles we will see a drop in demand. Turns out Tina from HR didn't need that vibed out custom calculator webpage after all.
whynotminot 2 hours ago [-]
Yeah I suspect the “average Tina” will have a top out on usage. Almost the same way that the average house only needs so much electricity before there’s not much more wiring up to do. Only so many lamps and TVs the average house needs.
But corporations? Government? Universities? Militaries? Biomedical research? I suspect the ceiling on their consumption of intelligence is much much higher than Tina’s.
cmiles8 4 hours ago [-]
There is a use for older GPUs, but the chips do have reliability issues. Turns out running chips hard 24/7 causes them to burn out. So it’s less will they be “useful” a while from now but more will they still work.
Folks are leaning it’s like these LED light bulbs that say they’ll last 15 years but after 2 years they start flaking out at a high rate.
Separately, the broader point is still directionally correct. We’re still using fiber from the .com network infra implosion today and will for a long time. The deprecation cycle is incredibly long. The increasing reports of GPUs sitting on shelves because there aren’t data center to power them is very concerning considering all the high interest debt being used to buy what is ultimately a rapidly depreciating asset. Financially, that’s a disaster waiting to happen.
booty 4 hours ago [-]
Turns out running chips hard 24/7 causes them to burn out
I don't have direct experience running CPUs or GPUs 24/7, but when I was looking into acquiring used GPUs I looked into this and it seemed like "burning out GPUs" was almost entirely related to the thermal paste cooking off and other heat-related issues. The tribal wisdom seemed to be that GPUs could run pretty much indefinitely if you kept their thermals under control and reapplied the thermal paste every N months.
Hopefully somebody with direct experience can correct me if I'm wrong...
robochat 4 hours ago [-]
Which is almost exactly the same story that applied to the flaky LEDs - bad thermal designs leads to the LEDs or their driver circuitry breaking.
lp92 2 hours ago [-]
It's amazing how many things fail due to bad thermal design.
NBJack 4 hours ago [-]
Fluctuations. If the use of the GPU is steady state, I think you're right; the thermals would be consistent. However, the reality is oftentimes loads come and go, and the rise+fall+rise of temps can lead to structural problems due to thermal expansion. Micro fractures in solder, thermal paste degradation, sagging PCBs, etc.
ActionHank 4 hours ago [-]
No, take a look into people who bought used GPUs that were from crypto farms. It's not just thermal paste, the chips become unreliable because of degradation of the circuits etched on the chips themselves.
4 hours ago [-]
lostmsu 2 hours ago [-]
My Titan Vs that were running non-stop since 2019 for 4 years and then on and off until mid this year exhibit no signs of any kind of issues. I just sold both because they don't have enough VRAM for LLMs.
4x3090s bought in 2021 also still work flawlessly (although they were permacapped at 250W to avoid fan noise).
4 hours ago [-]
deadlast2 6 hours ago [-]
If you use opencode for your AI work or becomes affordable. To the point where you would reasonably build applications with intelligence the LLM version. I think in the near future the intelligence will be free and there is no business in free. I know it is a big statement but if you look around there is already a large amount of free models of gradually increasing strength. For most applications you don't need a gpt6
smashspectacle 6 hours ago [-]
Ed Zitron is going to become the Simpsons of ai critiques...
pllbnk 5 hours ago [-]
Lots of respect for him for staying unwaveringly principled while knowing when and how to adjust his expectations and predictions. He is really well informed and it's something else to watch him speaking live on the podcasts and be so specific with numbers. He really knows his stuff.
His track record of prediction on this topic is stunningly bad, and he's made a job out of being counter to the AI hype, so he seems likely unable to honestly update his beliefs to adapt to changing evidence without jeopardizing the attention people pay to him.
pllbnk 4 hours ago [-]
Predicting global events is next to impossible. Don't look at the predictions, as his prediction game has slowed and he has recently admitted that he was wrong on his predictions. Look at his exposing of financial shenanigans which let the bubble grow.
ericd 4 hours ago [-]
The specifics are tough, but the rough shape isn't impossible. But it's hard, which is part of why most people try not to do it publicly with much confidence, lest they look like they're not worth listening to on the topic. Like, why are we listening to his opinions on this when he so clearly isn't an expert, and so many of his past opinions on it have been so far off?
And I think a big part of why you shouldn't listen to him is that he's said repeatedly that he doesn't really use generative AI outside of occasional experiments - he hasn't seen its potential enough to bother using it himself beyond some initial tests. Which is fine, but I think it also means that he likely doesn't have a good intuition of the likely future demand, which is pretty key in figuring out whether debt loads are likely to be sustainable or not. Investing is fundamentally about trying to predict the future and putting your money into the things that will likely be successful there. And he's demonstrably been terrible at predicting the future of this stuff.
saagarjha 5 hours ago [-]
He really does not.
sevenzero 5 hours ago [-]
Good point, very well structured argument here
empath75 3 hours ago [-]
"A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines. With consistency a great soul has simply nothing to do. He may as well concern himself with his shadow on the wall. Speak what you think now in hard words, and to-morrow speak what to-morrow thinks in hard words again, though it contradict every thing you said to-day" Ralph Waldo Emerson.
It would be one thing if he once admitted that he was wrong and adapted his world view to account for new information, but he just doubles and triples down.
jdietrich 6 hours ago [-]
Siri, what's the spot price for an A100?
4 hours ago [-]
anu7df 4 hours ago [-]
GPUs may last 6 yrs or more. Fiber vs GPUs is however not the right comparison. Fiber vs Open models are. Open models have mostly caught up with frontier models. Now, with a reasonable harness and tools they can easily be useful for vast majority of users for a decade or more.
Good4boothee 4 hours ago [-]
There are multiple YT channels proclaiming Russia is finally collapsing/losing war with Ukraine, huge "It is happening" in thumbnail; few of them put out a variation of such video every week, not even bothering to excuse their previous, wrong prediction.
And any reporting related to AI bubble is exactly as terrible. I might as well just check Bloomberg/FT/QZ, news there will be devoid of technical details and still more useful than those endless armchair-experts' speculations.
saidinesh5 5 hours ago [-]
GPUs might be 3 years but the rest of the hardware: processor cores, ram, storage can power other cloud applications for a decade or more no?
ericd 4 hours ago [-]
Except the GPUs last a lot longer than 3 years - A100s are 6 years old at this point and still in heavy use.
adrian_b 4 hours ago [-]
Some GPUs may easily last more than 10 years of light use, but I am not aware of public data about the reliability of datacenter GPus, which would give the percentage of A100s that die during the first 3 years of 24/7 use and during the first 6 years of 24/7 use.
I doubt that many of the 6-year old A100s have been in use 24/7.
ericd 4 hours ago [-]
In this paper, they examined a cluster with 448 A100s for 2.5 years at ~50% utilization, 4 were replaced over that time, about 0.36%/year:
I don't want to argue here, but once models are trained, using them is quite cheap. Also in the last few months most improvements I've seen are in the smaller sized models, which might make using models even cheaper.
Driver4732 6 hours ago [-]
If the models are cheap to run, while good for customers, isn't that a problem for hyperscalers and all the investment poured into the AI data center build out?
jsnell 3 hours ago [-]
No. If models become better, the natural expectation is for hardware to get more valuable, not less. (The hardware that was previously able to achieve X can now do 2X. Why would that make the hardware worth less?)
Likewise if models become commoditized, it's raising the value of the existing hardware (you'd expect the hardware owner to keep a bigger slice of the surplus when that's the scarce component).
For model improvements to be bad for the value of hardware, what you'd need is for demand for intelligence to be inelastic. But everything we've seen so far suggests that is not the case: the demand for intelligence (whether human or artificial) is highly elastic both in price and in quality.
skeezyboi 4 hours ago [-]
its a false equivalence.... the dotcom boom wasnt just a fibre rollout, it was all the internet company startups that were overpriced and crashed. no doubt some of the hyperscalars will get busted out, but the models and LLMs are here to stay. just like dot com websites didnt disappear after the dot com crash, neither will AI transformer models
BryanBigs 5 hours ago [-]
All of this spending only has a chance of paying off if video generation takes off.
KevinMS 5 hours ago [-]
> using them is quite cheap.
until the other guys comes out with a slightly better model, then using yours wont be cheap enough.
nisegami 6 hours ago [-]
There's this weird tension where small models are getting a lot better, but the gap between the frontier and "what I can run on my gaming GPU" at home isn't necessarily getting smaller. That said, Qwen-3.8-27B-Q3 runs on my 9700 XT and performs well enough for tool calling in agents, which is a critical floor in local model performance imo.
TonyStr 5 hours ago [-]
Do you use qwen for any coding? Do you make a frontier model orchestrate it? Are you running with llama.cpp? Sorry for the barrage of questions, but I've been interesting in local AI for a while, yet I've struggled to set up anything usable.
esseph 5 hours ago [-]
Just go download and install lemonade server. Open web ui for front end. There are mobile apps to connect to it.
My Qwen testing has been with Hermes trying to do mundane (to me) things that hosted models understandably refuse to do, even open-weight ones like Deepseek V4.1 Flash. Example: I tried to get it to find a torrent for $tvshow and add it to qbittorrent (connected to hermes via MCP already). It certainly tried and I think frontier models would have done better. My biggest issue was context space more than anything else. I had to drop down a quant level to increase it to 128k. On 64k it was constantly compacting.
mono442 5 hours ago [-]
Another thing to pay attention to is the margins of hardware companies like nvidia or micron. Hyperscalers are spending huge amount of money but the money mostly goes to the profits of hardware companies.
https://artificialliabilities.com
Bears are gonna eat.
My thinking is there wouldn’t be downward pricing pressure on data center compute if utilization was actually this high - it would mean providers could jack up prices and walk with larger margins rather than burn money for acquisition, right?
Plus the market wouldn’t increase the price of borrowed money if utilization was that high; they’d see the same signal. But the cost to borrow money for datacenters has roughly doubled year on year (might just be for riskier ventures? Buildings? Is this aggregate?) - those two signals wouldn’t disagree, it’s nonsensical (I mean, right?)
Edit: it’s just “the buildings”, and it makes sense: maximal utility comes from installing compute in a building already setup to house compute (cooling, infra, power). I actually take this as a negative signal: we’re seeing downward pricing pressure already, AND new projects aren’t fully booked, AND money is now twice as expensive. Somebody’s going to lose their shirt unless OpenAI’s Omnissiah comes online and converts us all.
The real question IMO is building capex. Those depreciate over a longer timeline, but if demand were to drop in 5 years and we have an excess of datacenter space, that's going to hurt. We're approaching the physical limits of what's possible with silicon and fiber optics. Unless there are new architectures or break throughs in algorithms, the only way to make AI more productive today is to build larger clusters, or much longer training/RL runs.
- The factories don't stop working after 3 years. They can keep making GPUs.
- The AI boom isn't just about GPUs. There have been massive investments in CPUs, RAM, networking and routing components, physical data centers, power plants, cooling, and even fiber (the thing you yourself claim lasts decades). None of this is going to disappear in a couple years.
Volta generation prices are collapsed, so GPU depreciation is currently around 8 or 9 years
Once there is enough, or even a glut, it's over.
Is intelligence like for example food where if I get my daily calories I’m good? Or is there no limit to the demand for intelligence. If there’s more available, will we just keep using more? Forever? It can’t be limitless right?
But where’s the limit because right now we’re consuming every GPU hour we can.
But corporations? Government? Universities? Militaries? Biomedical research? I suspect the ceiling on their consumption of intelligence is much much higher than Tina’s.
Folks are leaning it’s like these LED light bulbs that say they’ll last 15 years but after 2 years they start flaking out at a high rate.
Separately, the broader point is still directionally correct. We’re still using fiber from the .com network infra implosion today and will for a long time. The deprecation cycle is incredibly long. The increasing reports of GPUs sitting on shelves because there aren’t data center to power them is very concerning considering all the high interest debt being used to buy what is ultimately a rapidly depreciating asset. Financially, that’s a disaster waiting to happen.
Hopefully somebody with direct experience can correct me if I'm wrong...
4x3090s bought in 2021 also still work flawlessly (although they were permacapped at 250W to avoid fan noise).
His track record of prediction on this topic is stunningly bad, and he's made a job out of being counter to the AI hype, so he seems likely unable to honestly update his beliefs to adapt to changing evidence without jeopardizing the attention people pay to him.
And I think a big part of why you shouldn't listen to him is that he's said repeatedly that he doesn't really use generative AI outside of occasional experiments - he hasn't seen its potential enough to bother using it himself beyond some initial tests. Which is fine, but I think it also means that he likely doesn't have a good intuition of the likely future demand, which is pretty key in figuring out whether debt loads are likely to be sustainable or not. Investing is fundamentally about trying to predict the future and putting your money into the things that will likely be successful there. And he's demonstrably been terrible at predicting the future of this stuff.
It would be one thing if he once admitted that he was wrong and adapted his world view to account for new information, but he just doubles and triples down.
I doubt that many of the 6-year old A100s have been in use 24/7.
https://saurabhjha.one/pubs/SC2025/paper.pdf
Likewise if models become commoditized, it's raising the value of the existing hardware (you'd expect the hardware owner to keep a bigger slice of the surplus when that's the scarce component).
For model improvements to be bad for the value of hardware, what you'd need is for demand for intelligence to be inelastic. But everything we've seen so far suggests that is not the case: the demand for intelligence (whether human or artificial) is highly elastic both in price and in quality.
until the other guys comes out with a slightly better model, then using yours wont be cheap enough.
https://lemonade-server.ai/
No.
>Do you make a frontier model orchestrate it?
Also no.
>Are you running with llama.cpp?
Yes, on Arch Linux.
>Sorry for the barrage of questions
No worries.
My Qwen testing has been with Hermes trying to do mundane (to me) things that hosted models understandably refuse to do, even open-weight ones like Deepseek V4.1 Flash. Example: I tried to get it to find a torrent for $tvshow and add it to qbittorrent (connected to hermes via MCP already). It certainly tried and I think frontier models would have done better. My biggest issue was context space more than anything else. I had to drop down a quant level to increase it to 128k. On 64k it was constantly compacting.