My understanding is that you run queries over a local database, right? Which means you already need to have all the stuff downloaded that you want to filter. That's fine for power users or apps with access to plenty of storage, but I want something more on-demand and with less storage required.
Discussion
we use an optimized binary format. 148,000 notes is about 94MB, which is smaller than a handful of cached profile pictures.
even then we can just compact the db and prune it if storage was ever a concern. the image cache uses way more data.
Nostr has millions (and someday billions) of notes you might want to query based on arbitrary criteria. Feeds over a local database have the same problem any "global" feed has on nostr. If you don't know where to look, you have no guarantee you have everything you need. At some point, requests have to be built to fetch data from relays. Otherwise, you have to either sync with the entire network, or arbitrarily limit your feed to data at hand.
you don't need everything, you only need stuff that you're interested in or things in your WoT neighborhood.
for everything else there's nip50 search.
That's the kind of arbitrary limitation I'm talking about. Why not include nip 50 search in feeds? What about feeds powered by third party services that index the whole network for content outside your wot? I don't see why that stuff should a priori be excluded from feeds. It's fine to implement fast local feeds, but it should be an implementation detail behind a more generic interface, not the interface itself.
Just watched this, it illustrates what I'm talking about well. They even have feeds across protocols (RSS/bsky/activitypub)
I mean we support nip50 search feeds, i don’t know what point you’re making and why this is orthogonal to fast local feeds
I'm not sure what point you're trying to make either. All I said was that notedeck and coracle feeds aren't interoperable. Unless you're using my spec? https://github.com/nostr-protocol/nips/pull/1554
Placeholder for interop discussion here:
I was just replying to the claim that its not lightweight, i don’t know what the feed discussion is about. We just try to make sure the local relay has an api that is compatible with relays with extra functionality on top so you can do more advanced feeds locally and privately.
Any filter divergence just means we have to sync more data than we might need to sometimes yes. Extreme example is negentropy sync all data from your feeds in last 24h to not leak any query information to relays (privacy mode)
Sure, I guess we'd have to define what lightweight means. Having a syncing/cache component has its costs. But so does implementing set operations, so
looking at this, we have no concept of union/intersection/difference/dvm/wot(yet)/label so almost everything here is not useful to us