Replying to Avatar ben

running llama3 in the home lab. fired up an ide and installed a coding assistant plugin, pointed at the local llm. hot keys to add query and generate code, feels very fast.

I’ll probably always use vim as my primary editor but man some of these new tools are wild.

Avatar
⚑ Dee Kay βš‘πŸ‡ΈπŸ‡ͺπŸ‡¬πŸ‡§πŸ‡¨πŸ‡ΏπŸ‡§πŸ‡·πŸ‡¦πŸ‡Ή 1y ago

How much ram?

Reply to this note

Please Login to reply.

Discussion

Avatar
ben 1y ago

I’m running an 8b parameter model on a machine with 128GB ram and an 8GB gpu.

but that’s way more than necessary

Avatar
⚑ Dee Kay βš‘πŸ‡ΈπŸ‡ͺπŸ‡¬πŸ‡§πŸ‡¨πŸ‡ΏπŸ‡§πŸ‡·πŸ‡¦πŸ‡Ή 1y ago

Sweet!

Thread collapsed
Thread collapsed