Not quite sure what you mean.
Discussion
Reduce network usage. Instead of downloading the same duplicated info from 20 relays, we can download it from the fastest relays first and ask the others to send everything that is not included in the Merkle tree.
I'm not seeing how a merkle tree helps here is what I'm saying.
Are Merkle trees used in Nostr protocol? :-O
I first read your last post about trees and thought you were describing how you optimized. But after reading all the correspondence, I realized that you offer to use it.
It’s a good idea, but bloom filters would be more efficient. I give you a bloom filter that represents all events i have, and based on that you can send me back all events you know aren’t part of my bloom filter. The reason I’m suggesting bloom filters is because they’re much more space efficient.
I would extend it, that if you filter an already known note out, you could send a 1 of 0 if you have it without sending the note itself, so we will still know which notes are stored on which relays.
i believe this is how strfry relays handle syncing with other relays
https://github.com/hoytech/strfry#sync
`uses the yesstr protocol and performs a merkle-tree set reconcilliation against the specified relay.`
How heavy this is? Any chance of turning their implementation into a NIP and add into the regular filter options?
yeah ideally this would be a NIP for clients to efficiently deduplicate data from relays. it’s actually pretty efficient, and #[5] seemed to be speaking highly of the performance when relay.damus.io was switched over to strfry.
Correct me if I'm wrong but this seems to add overhead for no reason versus just sending the event IDs of what you have. Why add the overhead of the tree unless you want proper inclusion proofs (which would require merkelizing all events you have as a relay).
The hope was to tmget the tree or a bloom filter to reduce the amount of IDs. Amethyst generally has 50,000 IDs in the local database, which would make it a 3mb upload at every new filter request to a relay.
Merkelizing IDs though is just going to add more data, or if you dont send the whole tree add resource demands for relays to hash the tree from inputs.
How Merkel trees helps here? Merkel trees helps to identify if an item belong to it, but I don't see how it will help to fetch results
Put all the IDs you have in the tree. Send the tree to the relay. Now give me everything that i do not already have.
Is there a proposal for it already? Would love to see how that performance works on the relay side, I guess it would be an inverse proportion with the IDs array size
Certainly the date ranges filtering we have nowadays are insufficient on a network where the consensus points to zero. And there is no much proposals for relays on that sense.