Nostr Web Client

Frank 1mo ago 💬 7

#grownostr #bitcoin #llm #asknostr

My LLM server build. Running open source free LLM’s.

(5) 3090’s giving me 120gb VRAM

64 Core Epyy 7V13 processor

512gb ddr4 at 3200Mhz (8x64gb)

ASRock ROMED8-2T motherboard

(2) 4TB 990Pro Nvme drives

(2) NVLink Bridges (Both 3 slot spacing)

1600w Seasonic Prime Platinum PSU

1500w Corsair Platinum PSU

Modified open air case. Had to drill and tap (thread holes) sometimes to make it fit perfect. Have powered risers cables I will likely switch out soon. Room to grow 3 more cards, possibly 4090 added in that set. Long fan arrays not in picture running for cooling. Already have Noctua fans under GPU’s extracting heat. Everything runs cool, quiet and strong.

Reply to this note

Please Login to reply.

Discussion

kidwarp 1mo ago

Reminds me of when I was mining bitcoin with gpus…

ToBeDiscussed 1mo ago

Finally we can reuse the hardware we used to mine shitcoin nr.1

Frank 1mo ago

Never mined gpu shitcoins. Never bought shitcoins. But I love self hosted open source LLM’s.

ToBeDiscussed 1mo ago

congratulations on avoiding the pile of shit.. was fun at the time but your right. Self hosted open source llms are the way to go.

someone 1mo ago

What kind of percentage do the nvlinks impact in speed?

Frank 1mo ago

15-25% advantage on tokens/s

The higher % on big 405B models

The lower % on 70B models

I feel with my set up, I have an extra 3090 card almost, because of them.

Petulant 1mo ago

nostr:nprofile1qqsd73uwehl7j8deq356ankc6k3nal927afrrw6dw688zyvyf9gs0fcpzamhxue69uhkv6tvw3jhytnwdaehgu3wwa5kuegprpmhxue69uhhqun9d45h2mfwwpexjmtpdshxuet5y42hjl I just read your note and then this one. I think this guy could help you with hardware specifics for running LLMs.