Nostr Web Client

Is there hardware specific for AI that I can buy and put on my local network?

Not saying like an NVIDIA DGX, but something that is better than a regular computer.

I have an RTX 4080 and it works as well as it can on consumer hardware without bankrupting yourself , I'm guessing anything over 3070 would work well

Reply to this note

Please Login to reply.

Discussion

TKay 2w ago

What model size do you feel you can run comfortably?

ABH3PO 2w ago 💬 2

8-18b run very easily alongside other stuff,

18-30b run but I have to close other graphics and ram heavy stuff while using them.

Above 30b is a hit and miss and often gives my PC a seizure.

TKay 2w ago

Thanks. Problem is that I do t have a desktop computer. So I was hoping maybe I can get hardware that would just sit on the network with a vpn.

I’ll check and see if I can get my hands on some used hardware.

ABH3PO 2w ago 💬 1

It can run some smaller llms even with an integrated GPU or purely on CPU, something like llama:3.2:3b its also useful and you'll get a feel for things before you can run the larger models.

Richi 1w ago

That is what I do on my laptop!