Replying to Avatar ponymontana💜⚡

who need this
Avatar
jb55 10mo ago

local ai nerds running frontier models in their house

Reply to this note

Please Login to reply.

Discussion

Avatar
ynniv 10mo ago

Tuning my 1TB RAM, 128 core EYPC for DeepSeek right now

Avatar
HoloKat 10mo ago

1tb ram?

Avatar
ynniv 10mo ago

DRAM, not GPU. It's slow, but can r1 q8_0. The Mac will be faster, but costs four times as much

Avatar
HoloKat 10mo ago

How did you get 1tb dram?

Avatar
ynniv 10mo ago

https://digitalspaceport.com/how-to-run-deepseek-r1-671b-fully-locally-on-2000-epyc-rig/

Avatar
jb55 10mo ago

I have an erection

Avatar
ynniv 10mo ago

nostr:note1q47yfl7wpc29ma2adskj99l2ukrvgm70u7a5cw88rvz2pya4ds7s6wun6n

Avatar
ynniv 10mo ago

That's two threads per real core on two sockets with 64 cores each. Inference is memory controller limited though, so the simpler, cheaper DigitalSpaceport build seems better

Avatar
jb55 10mo ago

#unixporn