Replying to Avatar Melvin Carvalho

Ollama is an open-source project that serves as a powerful and user-friendly platform for running LLMs on your local machine.

https://ollama.com/library/llama3.1

Avatar
Laan Tungir 1y ago

I started using Ollama 2 days ago. Pretty easy to get up and running.

A little disappointed though at the size of the models I can run at a useful speed.

405b is completely out of my grasp.

Reply to this note

Please Login to reply.

Discussion

Avatar
Melvin Carvalho 1y ago

We are still early. Hopefully it will become easier in time, but the 405b isnt the only choice. Others are decent. What could be even better is running it as a service for sats. Openagents is still running llama 3 70B but they are moving fast.

da
Rand 1y ago

lotta things moving fast these days! t-y^