Nostr Web Client

The Ollama app was just updated to allow users to run a local 20b LLM with search capabilities and some tooling.

The models are somewhat big, but still great to see.

nostr:note1r8qzvx68myc8zvx3dahe923kwlamh4py2d52e4grju7fcnaeferqxatedk

These models are going to melt my i5 1235u + iris xe laptop

Please Login to reply.

It’s melting mine.

This is too much.

I have no context as to what kind of hardware can run these open models. Do you need a GPU?

Yes

The bigger the model, the stronger the gpu

Size doesn't matter. It's how you use it.

lol

It matters

I tried a 1.5b llama model if my memory wasn't weak, and it gave me a made up answer for updating solus linux

A higher nB model (7 or 8) gave me the correct result

Things doesn't come without a cost