After playing with local LLMs as coding agents for s few weeks here are some thoughts:

- Locally run LLMs are useful and nice companion that can help but they are like having very slow very junior companion

- They are private as they run on your machine but sometimes it feels like sitting in the privacy of your own bunker with an imbecile

- Running 7B or 14B models requires at least 16Gb of VRAM but those models are quantized which means in simple terms “not as good”

- For small tasks, explanations, docs creations they are useful if you can afford to wait but figuring out even medium complexity tasks is gonna make you throw your computer through the window

On the other hand, Claude is ridiculously good coding agent. Especially Sonnet 4. It’s like having an ultra fast medium developer working for you. It handles complex multi layer tasks ridiculously good but it comes with s price tag expressed in dollars for API and lack of privacy.