Nostr Web Client

Replying to

TKay

What model size do you feel you can run comfortably?

ABH3PO 1w ago 💬 2

8-18b run very easily alongside other stuff,

18-30b run but I have to close other graphics and ram heavy stuff while using them.

Above 30b is a hit and miss and often gives my PC a seizure.

Reply to this note

Please Login to reply.

Discussion

TKay 1w ago

Thanks. Problem is that I do t have a desktop computer. So I was hoping maybe I can get hardware that would just sit on the network with a vpn.

I’ll check and see if I can get my hands on some used hardware.

ABH3PO 1w ago 💬 1

It can run some smaller llms even with an integrated GPU or purely on CPU, something like llama:3.2:3b its also useful and you'll get a feel for things before you can run the larger models.

Richi 1w ago

That is what I do on my laptop!