M4 macs are becoming an interesting (and surprisingly cheaper) option for running local LLMs. They have lots of unified memory, integrated gpus and neural cores that are pretty good for running local models.
Discussion
What models are you running? Am building my own machine running running local LLMs too (AMD/nVidia based).
mainly llama atm, but been playing with others. I want to try qwen nostr:nevent1qqs88zs80vrrndpns2l88hxdgaumg4hstnttth9jfzhxcejww7tjyzcpz4mhxue69uhhyetvv9ujumt0wd68ytnsw43qzrthwden5te0dehhxtnvdakqz9rhwden5te0wfjkccte9ejxzmt4wvhxjmcpzemhxue69uhhyetvv9ujuurjd9kkzmpwdejhgmvsqle
My son is looking at M4 Mac .. is it worth the price .. he is an AI dev ..
Yes. The base model M4 Mac Mini is one of the best deals on any computer right now.
Yeah, I just saw that video too, even just the new base model Mac mini looks great for a local LLM at your house - plus it doesn't actually use much power somehow (thanks Apple Silicon lol).
I'm most amazed by what my phone can do locally, and what my mom's lower RAM/weaker CPU phone can do locally
Inference is getting more memory efficient over time, and device memory is getting bigger. Exciting times.
I've ran the numbers running my local LLM with this and its really hard to beat with a custom built PC
Just upgrade hardware in 24 mo. and you'll stay at a reasonable "bleeding edge" of local LLM bare metal
It's quite difficult to beat $799 for such a dominant mini PC (24 GB Ram)
I yolo bought a 64gb model, going to use it as my backup bitcoin node + large model inference tasks
Brilliant🚀
M4 Pro is still a great deal if you want to swing the ~2M sats
Just realized you can get an m4 macbook with 128gb of unified ram. That would be wild for large model inference. A little more pricey but… hmm
Surely there's more you could be doing to increase the attack surface
Minisforum EM780 is cheaper, 32GB RAM, and actually mini by modern standards (small enough to fit in a pocket)
Care to share link?
Looks like the official store is one of the only places that's not saying it's sold out right now
might as well build a cluster