Replying to Avatar Vitor Pamplona

What if relays could take in an AI prompt as a filter?

"Build a feed with everyone going out in Boston today but are not from MIT and are over 30 years old."

The relay will keep sending posts for as long as the client is connected. It's like a mini party.

Then we could rank the most active prompts to create loosely coupled groups using kind 1s.
Avatar
mark tyler 2y ago

That would be awesome. Price per user would be quite high wouldn’t it?

Reply to this note

Please Login to reply.

Discussion

Avatar
Psilocyberbull 2y ago

Likely not. Used Gpt4all? You can download a ~5 gb model and run it locally on cpu without internet. Shit is wild and acclerating exponentially

Avatar
mark tyler 2y ago

Hmm, thinking about this, if we just save the embeddings vector for each post (the potentially tricky part) and profile, and then each query gets an embedding and then we find the post+profile distance - maybe all of that requires about as much compute cost as a single token generation.

On the one hand we need to remember that running a few prompts for nearly free isn’t the same as running thousands - millions per day for nearly free.

On the other, maybe these embeddings are quite cheap compared to token generation.

I wonder how much they cost exactly in compute