671B param deepseek r1 running at 11tok/s on your desk with two of these. $180 a month for 2 years to pay this off, about the same as a chatgpt pro subscription. It’s a bit slower but you get privacy. Hmm…
https://x.com/alexocheema/status/1899735281781411907?s=46 nostr:note1rn95d949hlzjuaef20pvwly36wy090wpnqaypsn3g6q2jmc2lc2qeucxkd
Hmm. In light of my efforts so far with the dual Epyc, that's not a bad way to go.
Not worth it. Rather use that money to buy Bitcoin. Pay monthly subscription, you could still avoid leaking privacy.
Buying that Mac is like you choose not to lump sum buying Bitcoin.
Also, it's an Apple product
Privacy not likely
Actually i think it’s significantly higher than 180/m, i was looking at the wrong model. Would be more like $800 per month from the price ive been seeing online. Yeah maybe not worth it for now. Will wait till these get cheaper.
Yeah, was just going to say those boxes are $10k *each*. A $2k Epyc is slow, but not that slow
epyc has faster inference then an m3 ultra?
No, but you can get a few tokens per second for $2k
I went with two sockets and 1TB of RAM, but haven't gotten twice the performance of this yet: https://digitalspaceport.com/how-to-run-deepseek-r1-671b-fully-locally-on-2000-epyc-rig/
Very, very tempting
Compare that to my Dell Inspiron 6000 ..realised and bought first hand in 2005 .. with 2 gigs of RAM .. runs 32 bits #Debian 12 ( can't handle GUI ..so only ClI mode :-)
Still available on eBay for 30 dollars .. just one time payment .. no installments 😅
Oh ..screen went bad .. so it works with external display these days 😭 .. a 2007 HP antique monitor :-)
Amazing
There's also this guy's build
https://xcancel.com/carrigmat/status/1884244369907278106
CPU only but for inference that should be fine.
