Replying to Avatar 0xtr

For the #LLM experts out there, which #opensource LLM model is the best atm? Don't know if it matters but I will be using it for programming mostly.
Avatar
Alex 1y ago

https://huggingface.co/collections/open-llm-leaderboard/llm-leaderboard-best-models-652d6c7965a4619fb5c27a03

Need a lot of hardware to run them though

Reply to this note

Please Login to reply.

Discussion

Avatar
0xtr 1y ago

> Need a lot of hardware to run them though

The models doesn't seem to list HW requirements, how do you know if your computer is beefy enough to run them? And arent't they pretrained so you'd only need storage space?

Avatar
roshii 1y ago

7B parameters models work fine with 16 GB RAM, getting ~3 word/s on Ryzen 5 5600X. Running on GPU helps a lot even with a low end one: getting ~12 word/s on RX6400 with 4 GB GDDR6.

I was not able to run models with more parameters, limited by RAM.