Replying to Avatar jimbocoin 🃏

Starting to run LLMs on my own hardware. These model files are often 4-40GB in size.

Where are the torrents? #asknostr
Avatar
d34e832d... 1y ago

Have you tried smaller models like Ollama for specific tasks?

Reply to this note

Please Login to reply.

Discussion

Avatar
d34e832d... 1y ago

Star coder is nice for testing and research

Avatar
jimbocoin 🃏 1y ago

Thanks! I’ll check it out 🙏

Avatar
jimbocoin 🃏 1y ago

Yeah, I’ve got llama 3 running via ollama. Downloading the bigger version now. Should be able to run it once my new RAM arrives.

But mostly, I’m thinking about going forward. I plan to download and try out a whole bunch of different models all of which are extraordinarily large data files. This seems like a perfect job for BitTorrent.