running llama3 in the home lab. fired up an ide and installed a coding assistant plugin, pointed at the local llm. hot keys to add query and generate code, feels very fast.

I’ll probably always use vim as my primary editor but man some of these new tools are wild.

Reply to this note

Please Login to reply.

Discussion

How much ram?

I’m running an 8b parameter model on a machine with 128GB ram and an 8GB gpu.

but that’s way more than necessary