Replying to Avatar 0xtr

> Need a lot of hardware to run them though

The models doesn't seem to list HW requirements, how do you know if your computer is beefy enough to run them? And arent't they pretrained so you'd only need storage space?
Avatar
roshii 1y ago

7B parameters models work fine with 16 GB RAM, getting ~3 word/s on Ryzen 5 5600X. Running on GPU helps a lot even with a low end one: getting ~12 word/s on RX6400 with 4 GB GDDR6.

I was not able to run models with more parameters, limited by RAM.

Reply to this note

Please Login to reply.

Discussion

No replies yet.