Nostr Web Client

Good morning, Nostr. Who's running local LLMs? What are the best models that can run at home for coding on a beefy PC system? In 2026, I want to dig into local LLMs more and stop using Claude and Gemini as much. I know I can use Maple for more private AI, but I prefer running my own model. I also like the fact there are no restrictions on these models ran locally. I know hardware is the bottleneck here; hopefully, these things become more efficient.

Hazey 2d ago 💬 2

I used local LLM exclusively, mostly for coding. Two used 24g 3090's which provides 48g of vram. It runs models up to 70b with very fast performance.

When inferring or training,

1. It uses a lot of power, peaking around 800W

2. It spins up the fans pretty loudly

I don't think it's necessary to go local for open source coding though. Maple, mostly gpt-oss-120 is great for that. I think it is necessary to go local for uncensored models or training with your own data, and discussing things that don't fit mainstream bullshit narratives.

Reply to this note

Please Login to reply.

Discussion

Ivan 1d ago

I got a 3090 I

might pick another. Thank you!

John 1d ago

Dual 3090s is (somehow) still the sweet spot for local