Do people have a good strategy for syncing content through nostr?

For Mutiny when we sync nwc, dm, etc we save the highest timestamp of the event we last synced, however this doesn't work the best because relays send us them in descending order so if the app is closed before we process all the events, we do not fully sync all the events.

Have been thinking of switching it to instead have it save the last timestamp of the last EOSE we received. To try to better ensure we sync every message first.

Does anyone else have a better strategy?

Reply to this note

Please Login to reply.

Discussion

I do since/until filters. If traversing back (newest first), I track oldest timestamp of event seen, and then use that as the until on next call, end if no new event ids seen before starting again at the head. If going forward (oldest to newest), I just reverse this with the newest timestamp of event seen.

Negentropy might be the best option for a complete sync. I haven't used it myself yet.

What is negentropy sync, where can I read up on it?

Negentropy reconciliation. It's available in rust-nostr, just call `client.reconcile(..)`.

I want to know what negentropy sync is, please shill me some nips

Implementation on damus is not complete afaik. Join here for latest:

damus.io/devchat

Not sure how up to date this is:

https://github.com/hoytech/negentropy

which version? I think strfry has two and the latest is most efficient, but not sure if it ever got merged

Weโ€™ve coded Negentropy in Go and would like to make sure itโ€™s up to date with the new version. Do you have a link for it? Is the unmerged version stable yet?

nextneg branch on strfry, i guess it was never merged, but it was the basis for this article (i think?)

https://logperiodic.com/rbsr.html

nostr:npub1yxprsscnjw2e6myxz73mmzvnqw5kvzd5ffjya9ecjypc5l0gvgksh8qud4

Protocol version 1?

nostr:note1f6ym3cfyjfzzf64f060aglcvzn2mr36z4g4ve5ep2uw6h6kcucts0t44hq

looks like it

Weโ€™re on 1.0 ๐Ÿ˜Ž strfry better catch-up before they get smoked.๐Ÿ‘จโ€๐Ÿณ

Thanks for the Damus 1.8 update. Threads are loading now!

The v0 since strfry not merged v1 yet (IIRC v1 is not compatible with v0). But the rust-negentropy implementation is ready for v1.

What does this do? Could you explain?