You can just filter the posts I've already downloaded. It's going to take ages until relays are capable of what phones are already capable of today.
Discussion
Yeah, the issue is that not a lot of posts are downloaded. Searching the past is easy. But finding real time info to build cool new feeds requires downloading the entire last week of the relay and then filtering things out.
I want to get this stuff from people I don’t already follow too though! A “For You” feed, which really blows up the amount of stuff I would need to download. Sounds like a relay or DVM task to me…
Feels almost incredibly hard to swift through everything, even just to get embeddings of everything without classification.
Even Twitter / grok can only do work on your own social graph afaik. It would either need an insane amount of money / compute or a few more years but maybe I'm wrong and you can get away with just crawling through some popular npubs and you're good.