Replying to 21823843...

I just tagged strfry 1.0.0. Here are some of the highlights:

* negentropy protocol 1: This is the result of a lot of R&D on different syncing protocols, trying to find the best fit for nostr. I'm pretty excited about the result. Negentropy sync has now been allocated NIP 77.

* Better error messages for users and operators.

* Docs have been updated and refreshed.

* Lots of optimisations: Better CPU/memory usage, smaller DBs.

Export/import has been sped up a lot: 10x faster or more. This should help reduce the pain of DB upgrades (which is required for this release). Instructions on upgrading are available here:

https://github.com/hoytech/strfry?tab=readme-ov-file#db-upgrade

Thanks to everyone who has helped develop/debug/test strfry over the past 2 years, and for all the kind words and encouragement. The nostr community rocks!

We've got a few things in the pipeline for strfry:

* strfry proxy: This will be a new feature for the router that enables intelligent reverse proxying for the nostr protocol. This will help scale up mega-sized relays by allowing the storage and processing workload to be split across multiple independent machines. Various partitioning schemes will be supported depending on performance and redundancy requirements. The front-end router instances will perform multiple concurrent nostr queries to the backend relays, and merge their results into a single stream for the original client.

* As well as scaling up, reverse proxying can also help scale down. By dynamically incorporating relay list settings (NIP-65), nostr queries can be satisfied by proxying requests to external relays on behalf of a client and merging the results together along with any matching cached local events. Negentropy will be used where possible to avoid wasting bandwidth on duplicate events.

* Archival mode: Currently strfry stores all events fully indexed in its main DB, along with their full JSON representations (optionally zstd dictionary compressed). For old events that are queried infrequently, space usage can be reduced considerably. As well as deindexing, we are planning on taking advantage of columnar storage, aggregation of reaction events, and other tricks. This will play nicely with strfry proxy, and events can gradually migrate to the archival relays.

* Last but not least, our website https://oddbean.com is going to get some love. Custom algorithms, search, bugfixes, better relay coverage, and more!

Did you see my article ?

https://habla.news/a/naddr1qvzqqqr4gupzp8lvwt2hnw42wu40nec7vw949ys4wgdvums0svs8yhktl8mhlpd3qqxnzdejxqmnqde5xqeryvej5s385z

Sorry I wrote it fast.

Reply to this note

Please Login to reply.

Discussion

Oh no, I missed this one, thanks for linking! This is a really good write-up and there are a lot of similarities with what I'm designing for strfry proxy. Like some of the comments, I'm not sure if partitioning on ID will be optimal in all cases. I can also imagine variations on pubkey and created_at.

Some variant of consistent hashing will also be necessary I think, for failure recovery, rebalancing, changing the number of backend relays, etc.