apple silicon gpu rising in benchmarks scores. having an rtx 3090-comparable gpu in a laptop with 64-128gb of unified ram is going to to be interesting…
especially on a “budget” compared to 64gb nvidia gpus which are like $30,000
large models require lots of gpu memory, so apple silicon seems like an increasingly interesting platform for the budding ai hacker. 
You benchmarking of you cooking something up for #damus
I hate apple and I will never forgive them for pushing for DRM at a pivotal time when they could have stopped it.
That said, this is some pretty good looking hardware.
Unified memory is amazing, but it's more expensive per GB and slower at inference than a box full of used 3090s.
It's been a long time since local metal mattered.
there's a convenience and power factor though. a box of gpus is annoying
Agreed, and I run both for different things. I'm not personally there yet, but no one expects to run a 405B model on unified memory.
why not, assuming gpu cores keep increasing?
also from what I understand its mostly memory bandwidth constrained? so its not all cores.
Right. Nvidia is ahead because they have higher memory bandwidth as well as computation. Apple is winning in the efficiency corner, which is how we get 4B models on phones (Apple Intelligence, but also LLM Farm). All of it is pretty fantastic, and I'm going to surf this wave as long as I can.
Oh, I didn't actually answer your question: laptop memory will always be limited because DRAM needs constant refreshing. Every GB installed reduces battery life.
