The hard part is to convert it into a Nostr filter to use with regular relays. The filters might not be able to represent the actual needs.
Running on the relay, though, the server can filter and order the events in any way it sees fit.
Discussion
You can just filter the posts I've already downloaded. It's going to take ages until relays are capable of what phones are already capable of today.
Yeah, the issue is that not a lot of posts are downloaded. Searching the past is easy. But finding real time info to build cool new feeds requires downloading the entire last week of the relay and then filtering things out.
I want to get this stuff from people I don’t already follow too though! A “For You” feed, which really blows up the amount of stuff I would need to download. Sounds like a relay or DVM task to me…
Feels almost incredibly hard to swift through everything, even just to get embeddings of everything without classification.
Even Twitter / grok can only do work on your own social graph afaik. It would either need an insane amount of money / compute or a few more years but maybe I'm wrong and you can get away with just crawling through some popular npubs and you're good.