For the #LLM experts out there, which #opensource LLM model is the best atm? Don't know if it matters but I will be using it for programming mostly.
Discussion
To stay up to date, I like the Leaderboard on https://chat.lmsys.org
Check for open source models in the License column. You can sort by Coding if required as well :)
Thanks!
Claude 3.5 Sonnet, I've been using PPQ.AI
I think Apple just rolled out a fancy new open source LLM tho.
Claude 3.5 Sonnet is proprietary tho
This finally got me to try a self-hosted LLM. Let me know if you find any good models. ollama was super easy to get running.
ollama works great for me too but I wanted to use GPT4All (https://github.com/nomic-ai/gpt4all) but had issues downloading new models, constantly getting errors without any explanation. I'll have to look more into it tomorrow.
Need a lot of hardware to run them though
> Need a lot of hardware to run them though
The models doesn't seem to list HW requirements, how do you know if your computer is beefy enough to run them? And arent't they pretrained so you'd only need storage space?
7B parameters models work fine with 16 GB RAM, getting ~3 word/s on Ryzen 5 5600X. Running on GPU helps a lot even with a low end one: getting ~12 word/s on RX6400 with 4 GB GDDR6.
I was not able to run models with more parameters, limited by RAM.