I'm trying to understand how to replace my not-very-private but useful ChatGPT 4.0 subscription with Llama 3.1.
ChatGPT translated the system requirements for https://llama.meta.com/ into slightly less confusing versions of "your beefy desktop is by far not enough".
So if I still need a compute cluster that would sit idle 99% of the time if I ran it just for myself, I'm kind of back at square one. I'd have to find a way to share these resources efficiently and privately.
Where can I use a powerful AI in a privacy preserving way? I want to pay with eCash and use it via TOR without any email or other accounts attached.
Been looking into this myself. It is possible to run AI without a GPU just not sure of performance. I think most private way is run it yourself.
I was looking into running a media server (hi-fi stackable) with a GPU to run Jellyfin so the GPU would also get use for that, and then possible to run a GPT that can be used by the whole family.
To my understanding, the cluster and GPU setup is for heavy duty operations. I suspect, things can be very well parallelized this way and running a single session on this setup would just not be resource efficient but you could fill your disks with Llama3.1 and use it at your discretion without GPU.
