Replying to Avatar Vitor Pamplona

Hum.. I can actually add a local LLM to Amethyst and make it reply automatically to every post you see from a template prompt.

"Reply to {post} with a funny, intriguing, snarky or less known take the topic. Make it short, 5 sentences or less, direct and casual."

Just scroll down and 10s of replies are sent. 🤔
Avatar
calle 10mo ago

How long does it take on your pixel (which model?)

Reply to this note

Please Login to reply.

Discussion

Avatar
Vitor Pamplona 10mo ago

4 seconds. But it's all hacked up with python shit everywhere. I need to explore a more native library. More to come.

Avatar
calle 10mo ago

The python shit is usually not the bottleneck, it probably uses native libs for the LLM stuff. Are you using the TPU?

Avatar
Vitor Pamplona 10mo ago

Frankly, I am just trying a bunch of stuff/libraries/demo apps to see where we are at with local LLMs. Most of the stuff I am seeing are just very poor ports of server runtimes, which is terrible.