#ai #llm #localllama #localllm #grownostr #nostr #gfy
Not your own locally hosted LLM? You are giving away your thoughts and ideas to corporations/governments, who pay for what and how you think.
https://video.nostr.build/a027a52098435f570e0340e42b93296b0b608dd462b1c3a57873c254b50289a2.mp4
This is pornography.
Storage currently is x2 990Pro NVMe SSD’s. The bridges for paired 3090 gpu’s help a lot, so instead of both boards traveling back down the PCIe lanes only one needs to, enhancing speed. Cards on these 3 slot bridges will be close to each other but not touching. I will have fans under the cards aiming up (hot air rises) to get rid of heat from gpu processors as fast as possible. I see a lot of fans. But i have built similar and it should be pretty damn quiet. 24GB of vram per 3090 x 6 or more is healthy. Avoiding overly aggressive quantization, opting for higher-bit precision to preserve model fidelity.
super cool. i am interested in how well you can harness these together! (software wise and stuff, and also gonna check out that mobo)
I will install necessary nvidia items, such as the cuda tool kit. I would get PyTorch. Install transformers. Utilizing Hugging face. Having a linux machine running all gpu’s as one and dumping a whole model into vram is not a hard. I do admit I use a ton of help from ai and reddit groups, etc.
As far as mobo, I was also considering an ASUS SAGE, as well as boards from supermicro. Threadripper and Epyc are the only processors that I think about. The best bang buck for the vram is key. A healthy amount of system ram is wise as well.
You’re a pedophile
Do you really need 1600W?
Numerous 3090’s are thirsty. But I plan on cutting back their power max usage. Not to mention the cpu and mobo
I'm off grid solar...
I could only make queries on a sunny day 😂
Off grid solar is badass. I would just make a less hungry machine and add some panels and batteries to your array.
Wow, very nice 👍
