Nostr Web Client

M4 macs are becoming an interesting (and surprisingly cheaper) option for running local LLMs. They have lots of unified memory, integrated gpus and neural cores that are pretty good for running local models.

https://youtu.be/GBR6pHZ68Ho

Ben Weeks ⚡ 1y ago

What models are you running? Am building my own machine running running local LLMs too (AMD/nVidia based).

Reply to this note

Please Login to reply.

Discussion

jb55 1y ago

mainly llama atm, but been playing with others. I want to try qwen nostr:nevent1qqs88zs80vrrndpns2l88hxdgaumg4hstnttth9jfzhxcejww7tjyzcpz4mhxue69uhhyetvv9ujumt0wd68ytnsw43qzrthwden5te0dehhxtnvdakqz9rhwden5te0wfjkccte9ejxzmt4wvhxjmcpzemhxue69uhhyetvv9ujuurjd9kkzmpwdejhgmvsqle

Ben Weeks ⚡ 1y ago

Just took delivery of an Asus GeForce 12GB 3060. Not sure if I’ll swap it with my daughters 3070 yet :-)

Ben Weeks ⚡ 1y ago

You use Hugging Face to try them out?

jb55 1y ago

Not sure what this means

liminal 🦠 1y ago

Its a repository for different models