- cross-posted to:
- technology@lemmy.ml
- technology@beehaw.org
- cross-posted to:
- technology@lemmy.ml
- technology@beehaw.org
You must log in or # to comment.
About the only upside to this that I can think of is that maybe, just maybe, programmers will start being more frugal with their memory usage. Everything else just sucks.
programmers will start being more frugal with their memory usage.
They 100% will.
Windows had started only “certifying” machines with 32gb as “windows ready” because they wanted the headroom to cram on AI features and use what we pay for instead of a data center.
But because no one was buying 32gb machines and a machine without that sticker will end up with Linux…
They just dropped it, and now they’ll have to keep optimizing for 16gb instead of just letting everything bloat. If they don’t, they risk every entry level machine making people eventually switch to Linux for a much better experience. Windows has to run on entry level or people don’t get sucked in, no one converts to windows later in life.
That’s a huge deal, if 32gb was the default for windows it would be the default for everything. Every game and design/modeling program would feel safe to assume 32gb was the norm. Like, imagine what chrome would do assuming there’s 32gb to play with.
Now they can’t, and they’ll know requiring 32gb will cut off some possible consumers.
It’s not like Windows is the good guys tho, it never should have been 32, and arguably should be 8gb forever, even if it required a lightweight mode. Most people would prefer a super stripped down Windows even with 64gb. Most of the bloat is useless, I could live without any updates after we didn’t have to quit to DOS to launch a game.
At this point i don’t see why we would need more RAM for basic OS or web usage tbh.
Like we’ve hit a point where things are covered. Obviously this won’t stop capitalists from doing capitalist bullshit though… and bloat is a thing.
Not going to disagree, but “web usage” is basically everything. You can literally run an entire desktop operating system from 2007 on a virtual machine emulated in JavaScript in your browser now.
If corporate lets them use Qt instead of Electron, yes. If not, we will continue to see memory usage rises all over the place.
I am quite amazed at your knowledge and foresight. Really cool!
I never assumed such a scenario where ram limit will limit our infrastructure too. What do you consider will happen with people who are actually needing ram but unable to afford? And that number reaches significantly higher volume, like 60% of population?
Like maybe an open source project of building ram seems feasible?
No, you only need 4gb to run Win11.
The 32gb referred to gaming PC’s
Everything is a Browser.
No chance.
Of course a chat program needs a 4GB electron app. That’s the bare minimum to match 1990s ICQ.
I remember moaning about icq gobbling 75k.
There are plenty of IRC clients that don’t use electron.
I think it was a comment on the use case for an app not really changing but the resources to run it bloating significantly… E.g. software running on a computer with MB of ram then and now the app alone requiring GB.
did you know Edge has a fucking task manager?
shift-esc to see the bullshit
It’s a fork of chromium, which has it.
AFAIK Firefox also has one.
Yeah so does ff
TIL, thanks!
That’s been my hope. Maybe get another wonder like Doom
Or just… Play Doom :)
wonder “like” Doom. Something that is efficiently created in the same capacity as Doom. Doesn’t even have to be a game.
Not most. Most only concern themselves with “developer productivity”.
“how many commits have you done today workerslave_96?”
Well, the current trajectory points to:
-
LLM capabilities topping out.
-
All the tooling around them not keeping up anyway.
-
Actual “AGI” being distant and completely unrelated to contemporary models.
-
Inference costs plummeting.
The last one is critical.
Right this second, you can run Minimax H3 on a desktop for the tiny fraction of the compute/RAM OpenAI Sora took. And it’s better.
In a month, it will be ~8X faster.
You can run DeepseekV4 flash, dirt cheap, and get what Claude was less than a year ago. And it’s gonna spread to every host out there, to systems like Cerebas ASICs that don’t even need HBM.
So… Even if you’re an AI acolyte. And we go with that for the sake of argument…
We don’t actually need all that RAM for hosting generative models?
I’m very interested to see what happens to all these datacenters over the next two-three years.
They spent all this money on something that’s gonna be cheap as dirt to run, largely run locally, and that won’t need GPUs once bitnet takes off, sooo… they can’t make money off that.
What happens then?
What happens to all those Stargate RAM wafers, and excess datacenters?
Deepseek v4 Flash is insanely good and basically free. Especially the latest model which I think is only available from China right now.
which I think is only available from China right now.
Nah, 0731 weights were released. I’m running it locally right this second.
I don’t know about Deepseek API specifically, but there are tons of places to get it.
You are Mr. moneybags or you’re running 2 or 4bit quantized.
I have a 3090, 7800, and 128GB ram from pre-rampocalypse. It’s not nothing, but there are definitely crazier home labs.
Deepseek V4 is native 4 bit.
I am running this quant, but the quantization loss is measurably low: https://huggingface.co/Downtown-Case/DeepSeek-V4-Flash-0731-128GB-RAM-IK-GGUF
The native MXFP4 is only a bit bigger.
A used 3090 alone is like what, 4-5k now?
Your setup isn’t really achievable for most now.
Lol what? I just checked, on eBay I can find a lot of offers between 800 Euros and 1300 Euros for a used RTX 3090. 4-5k would get you a brand new RTX 5090.
Okay, not as bad as I thought, there’s a guy selling a 4090 for “only” 2k. There’s nobody selling a 3090 in the local hardware forum. I’m not going to buy off ebay because if someone on ebay sells a dud, I can’t go back to punch him in the face.
5090s start at about 5200, that’s true. To be fair, the “4-5k” for a used 3090 was a bit hyperbolic, I’m just mad that they’re so expensive that my last 2 cars cost less than a USED 4090.
The 3090 is from 2020. It’s under $1300 based on eBay prices I’m seeing (which is around what it launched at).
Crazy expensive, but not $5K.
The 128GB RAM is madly expensive now, though.
You are 100% correct IMO for the AI stuff, just a question, what’s bitnet? Is it some supposed bitcoin-killer (because that’s not going to happen, IMO).
I also think we’re on the cusp of “having enough” personal computing power with the usual exceptions, take a bit, give a bit.
Our mobile phones are close to what’s needed for most people (8GB, 4k, gaming, …). Or so I think!
Bitnet is a catch-all title for ML models that use a specific mathematical trick.
If elements of a matrix are composed of only 1, 0, or -1, multiplying them is the same as as adding them.
That’s huge. LLM computation is basically all matrix multiplication, so if you replace that with simple addition, you reduce the computational requirements by orders of magnitude.
The catch is such models are hard to train effectively; its proven that it works, but research to get the technique usable and practical is still being done.
Personally, I suspect it’s unviable for many “dense” parts of models, but sparse hybrid bitnet models would be really cool.
I mention it because, if it takes off, suddenly the massive matrix multiplier accelerators we have for LLMs aren’t as useful. Chips with simpler architectures could get the job done, at least for parts of models that are bitnet.
Thank you! Ah yes that makes sense somewhat I guess. Trading cpu usage for memory usage, an old classic 😁!
Nibble * nibble look up tables could probably be fast too, it all depends on where you want the complexity I guess.
There are indeed some Bitnet LUT implementations, already.
It’s both smaller and faster though; the models are drastically smaller than FP16 at the same parameter count, as they’re just 1.5 bits instead of 16.
How much is lost from the precision drop is an open question, though, and the answer seems to be complex and finicky. I think it’s why big labs are hesitant to pursue it; they want to try safer, lower hanging fruit first. Also, it’s not particularly efficient on hardware (like AI GPUs) where most of the silicon is dedicated to matrix multiply performance anyway.
Lower inference costs will lead to more demand for RAM
Will it though?
I posit there’s a saturation point where there’s “enough” LLM in use, and we are not far from that.
The whole pitch from Altman and such is scaling models up. But that’s not working.
-
It should be noted that they haven’t become harder or more expensive to make. They just cost more.
More demand exists so they can raise prices and still sell out their product to someone, most anyone will take free money on the table. Would expect the prices to go down only if more fabrication plants come online in numbers enough to satisfy that demand with way more supply, or if the demand plunges (ex. AI bubble pops)
But it can take billions of dollars and years of work to build fabrication plants so new supply is slow to come online… and they want to be REALLY sure that they won’t become bagholders of epic proportion by investing that kind of money into plants only for a bubble to pop which would leave them with a lot of debt but only able to sell oversupplied RAM at very low prices
That’s a lot of words to say “illegal price fixing”.
They have fixed prices before but that looked different. See the circled area on the graph where there had been declining RAM prices but that decline suddenly stopped and became really stable? That period of 1998-2002 is where Hynix, Infineon, Micron Technology, Samsung, and Elpida were all found to have been fixing prices on RAM because all the supply they were bringing online was crashing the prices but they hoped to collude with each other to keep the prices stable rather than continuing to decline in price competition.
That coordination worked well for the way down (except when they got hit for hundreds of millions of dollars & euros in fees when it got discovered but eh), just a matter of agreeing on a number they all won’t go below. But how would they coordinate a continuous rise in lockstep with each other, that’s a LOT of phone calls anytime someone wants to buy? Why are many committing the billions of dollars to building new plants if they know that the only reason they’re making money hand over fist is avoiding selling product until it reaches a high price, why shell out money to bring supply up in that case? And why did their previous price fixing look only like a flat line if they were capable of having it go straight up? If they had that ability all along I don’t think you would have seen this somewhat sinusoidal graph with a sharply downward longtime trend for prices.
There’s already investigations and evidence that they are indeed price fixing, just check the Gamer’s Nexus video on it.
I just saw some of the video. Refers to a class action lawsuit and quotes from it heavily. The class action seems to not be alleging price fixing per se (meeting in a smoky room and determining price) but rather than they are colluding to all cut DRAM supply and profit from the price spiking since the demand is unchanged. Even to the point of forgoing profits.
It doesn’t seem all that convincing though so I don’t think it will go anywhere. Something similar was tried a couple years back that was shot down by the courts. It’s not illegal for competitors in the market to behave similarly if they consider it in their rational interest to do so, it’s only illegal if they are cutting a secret deal for it so you have to have some good quality “plus factors” to convince the courts that some shady deal is going on. If there are reasonable alternatives to explain the behavior other than collusion then your case is kind of cooked.
It’s easier if they have slack production that’s being left idle to point to (which still didn’t work earlier) but now they’re all trying to churn out as much HBM as possible for insatiable data center demand and are building new fabrication plants. The video said as one example that they have better margins on DRAM than HBM but while that may be true in the current state where everyone and their mom is focusing on HBM, one question that could be posed is would that remain the same if one of the major companies swapped focus? Like one of the three biggest gives up on chasing the AI money and stops making as much HBM to prioritize DRAM. That would cut the supply for the former and boost that for the latter, which would send prices up in what they just abandoned while crashing the prices into what they’re re-entering. They could secure a lot of market share in DRAM but that could be considered to be a sucker’s bet to sacrifice the foothold in HBM and enormous stacks of AI cash in exchange for cornering the market on an older technology. Also they are reaching agreements to make HBM at longer prices but for a guaranteed period of time which makes their margins on HBM weaker but also gives them confidence that they’ll be getting that smaller amount of money for sure even if we get an AI bust.
Moore’s law applies to all integrated circuits. You will find the inverse graph in this Wikipedia page.
Did you mean to reply to me or someone else?
Every stars are aligned. We are At the top level of capitalism greed. For decades tech companies have piled up as much cash as possible sucking every dime selling their product. Now they want to rent their service instead. The best to do that is to host the service on a super computer, the user having only a low power terminal. They deliberately crippled the os to use the maximum ram and said we need to update our computer for ai and other things most don’t want, don’t need and don’t have the money to pay for it. The best move is to exchange their huge pile of cash to drain the whole market to build data centres. They now want everyone to become dependant on ai and pay them a significant part of our salary to satisfy their shareholders.
This. These companies (Microsoft, Google, etc) are using OUR computer’s Ram, Hard drive space and CPU time to run their very own Skynets and making us pay for the privilege.
It’s the Golden Calf all over again.
Then it’s time to let them go bankrupt and trust bust them to oblivion
They say those who actually made fortunes during the gold rush were the ones selling the pickaxes.
Maybe I should get into the guillotine business…
Gordon Moore is turning in his grave
The new Moore’s Law: the amount of RAM you can afford halves every six months.
Gonna upgrade my server with another 16gb of ddr3 soon.
£30.
deleted by creator
Just wait till the bubble bursts.
RAM prices haven’t been normalised they are still rising.
Surely this will make them finally realize it is unsustainable to demand consumer to upgrade their hardware every few years?.. Right? Right??
Correct, so now they will move to computers as a cloud service.
They will eventually be forced to
There are only a handful of things billionaires hate more than dropping prices of anything.
The price of creating and shipping the ram has not gone up, tho. People just saw the rarity and decided to jack the everloving shit out of the price of stock they already held.
