OpenAI buying tens of thousands of Macs for AI is the funniest possible plot twist: spend years convincing everyone the cloud is the future, then discover the hottest AI infrastructure is basically a warehouse full of Mac minis. Somewhere, an IT admin is whispering: »I told you so.«
I’d be interested in buying one as a headless server, but a lot of the FLOSS software I’d like to run seem to prefer x86 processors.
Truly? Even with their translation layer? I forgot what’s it called.
The translation later is called Rosetta 2 (Rosetta 1 having been for the PowerPC to Intel transition), and it looks like it’s mostly going away next year, so buying a Mac with the intention of using it would probably be a bad idea at this point.
I was thinking more about things that are written for Linux and not necessarily ported to macOS. When I’ve tried things in an ARM VM there seems to be a lot less software compiled for ARM.
Smells like 'Open’AI is digging another moat where there is none, like it did with the memory wafer deal that started RAMageddon. Heaven forbid people actually being able to run local models, can’t run local models if 'Open’AI has bought all the capable smaller machines, taps head. Fuck SamA and his market cornering with a rusty chainsaw. I guess they think all they have to do is keep that moat somehow until IPO and then it’s somebody else’s problem, they and their investors cash out. Dear Goddess I hope it blows up in their faces.
My new gaming PC that I bought about 18 months ago went up in value by 75% based on its RAM and GPU price and while I’m relieved I got it when I did, that’s pure insanity that a PC would appreciate in value
I overbuilt built my NAS/container server for $1262 excluding the price of the rack chassis, PSU, cooler and two 1TB NVMe SSDs I cannot recall the price I paid for. B550 board, Ryzen 5, four 10TB Red Plus HDDs, Intel Arc A380, 64GB of DDR 5 ECC. Those listed components are now $3381. When or if you can even find them. The Arc is unobtanium as far as I can see. Of course the RAM and HDDs went up the most in cost, but it’s still like $65 and $75 more for the mobo and chip.
Whoop, decided to check and I found the SSDs: I paid $60 a pop for Samsung 980 Pros and they’re $240 now. So $1382 then vs $3861 now. Almost triple. The bubble can’t pop soon enough.
I’m convinced that OpenAI buisness model is to create AI bubble, to sell its “reserved memory wafers” and now Macs, at the top of ram extortion pricing moment.
That same thought just occurred to me: hype a McGuffin and get investors to lend you insane amounts of money which you then use to buy up a marker, causing artificial scarcity, then sell at massive profits.
Local 27b models are good enough for most tasks.
Can’t wait to buy one of these from Ebay for 10% of the price next year.
Yeah no way they will allow any of this hardware to go back onto the market. Anything they dont use anymore will be destroyed.
And that’s really why they’re hoarding them.
https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant
I’m running gwen 3.6 with 131k context window on a 3090, it’s fast enough and about as good as pay to play Claude at work.
Upgrade that to 3.8 as soon as your hardware allows (and your use case makes sense). 3.8 is quite a bit more rational.
I plan to once there is a version with turboquant and MTP as that huge context window is key.
A used 3090 is like 2-3k though, IF you can find one :| A month of claude is like 20 EUR. A month of opencode go is half that, but you get less usage.
Any 27b Model you can currently recommend for a 16gb AMD ? Mostly coding tasks but not exclusively.
There is a way. There was a post yesterday on exactly this, let me find it… https://lemmy.world/post/51283416
Thx I’ll give this a try!
9060xt 16gb is the most cost effective new GPU, but if you’re going used look for a V620 on eBay. It’s a 6800xt chip but in server form factor GPU with 32GB vram. Can be a bit of a pain to set up but by far the most cost effective option IMO
V620s were a good deal when you could get them for $350, now they’re $700+ and no longer a good deal.
Oh wow you aren’t kidding. The dude I bought from on eBay @ $350 in February is sold out now. Guess I retract my statement.
This AI pricing is wrecking every deal on the market lol
There just aren’t any good deals any more. Prices for everything have gone crazy in the last few months. For coding LLMs the cloud services may now be the least worst value, by design, until they hike the prices.
That said, I still just paid way too much for a used graphics card so I could do many things locally, because I just don’t want to give the likes of Sam Altman a single penny.
Thanks! actually have a 6800xt already and was asking for model tips. I saw my question was easily read as asking for GFX card tips, edited.
For your hardware, the VRam is not enough to run 27b but, I’d recommend Qwen 3.5 9b for image / text to text.
And I’m planning to experiment with Qwen 3.8 9b for text to text.
4_k_m quantization is the sweet spot for performance and ram usage.
Also, I find Llama cpp is better than Ollama in terms of performance.
Thx for the tips!
cant wait for their debt calls and they have to sell their inventory to make payment.
Intel must be kicking themselves for not using their own fabs for Arc, AND divesting from Micron, Optane, and HMC.
Apple and others must be too, for snubbing Intel and Samsung.
They would be making money hand over fist, and relieving the market at the same time.
Well, this may explain why my work computer has been on back order for the 4 months that I have had my job. Still have no idea when I’m actually going to get it.
sigh I just wanna upgrade my video card, man
They’re the worst kind of scalpers
y’all realize this is what a hostile takeover looks like, right?
Are you talking about a stocks takeover? Because I fail to see how buying up hardware has anything to do with it.
hostile takeovers are more than just stocks.
it can include supply chain attacks against over leveraged companies.
