What if relays could take in an AI prompt as a filter?
"Build a feed with everyone going out in Boston today but are not from MIT and are over 30 years old."
The relay will keep sending posts for as long as the client is connected. It's like a mini party.
Then we could rank the most active prompts to create loosely coupled groups using kind 1s.
Great idea!
This is the type of stuff that makes me so excited for nostr. Fantastic idea.
That would be awesome. Price per user would be quite high wouldn’t it?
Likely not. Used Gpt4all? You can download a ~5 gb model and run it locally on cpu without internet. Shit is wild and acclerating exponentially
Hmm, thinking about this, if we just save the embeddings vector for each post (the potentially tricky part) and profile, and then each query gets an embedding and then we find the post+profile distance - maybe all of that requires about as much compute cost as a single token generation.
On the one hand we need to remember that running a few prompts for nearly free isn’t the same as running thousands - millions per day for nearly free.
On the other, maybe these embeddings are quite cheap compared to token generation.
I wonder how much they cost exactly in compute
Interesting idea.
This actually isn’t that far-fetched. One could calculate semantic embeddings for each arriving note and store them alongside in the local db. These embeddings are typically arrays of floats in the size of 300-1500 entries each. The machine-learned algorithm has been pre-trained to encode similar meaning sentences into numbers that are “close” in the vector space of the embeddings.
Then, for a feed prompt, one calculates the semantic embedding of the prompt, and can query the embedding database for the top-K closest notes that match, or in real time, filter out a stream of notes based on semantic similarity as they arrive.
All this exists and only requires commodity lower-end GPUs (or even just CPUs, but it is slow).
Kind of an obvious application in retrospect.
