This is not useful at all. A creator or a business is trying to reach people who will pay one way or another, not reach people who run indexers that can't see a fraction of a fraction of a fraction of the web.
Reach is the name of the game here, and running your own indexer doesn't help you (your social graph is most probably boring as fuck, and limited relative to your interests and needs), and doesn't help anyone else because you won't see their content.
People will end up using a bunch of different types of indexer.
I would find it pretty useful to have at least one indexer that searches an index of things that my friends, or friends of friends, or community, think is useful/relevant to me... while limiting businesses that pay money to push their product/narrative at me.
Again not realistic, the cost of indexing is fixed (going through massive amounts of data, that is even if you already have it and don't have to crawl it first), so assuming that we will have artisan indexers doesn't make any sense, it is expensive as fuck!
Idk about the economics... but surely I'd only need to keep a [relatively] up-to-date index of my own designated files. I could rely on signed indexes of my first/second/n^th degree connections, cached onto my homeserver, for wider reach.
Again, how are these going to be created? Who puts all the work to download all the data and index them? If you only care about like a 1000 blogs, then fine you can always fetch latest updates and read them all like RSS works.. but that has nothing to do with search or algorithmic feeds... These by definition require unbounded number of data you have to crawl through to find the best of what you are looking for
I guess its a trade-off?
I consume a lot of video and audio content, but I'd get by with a transcription of podcasts and informational videos.
The info that's really important to me is predominantly text: Health related stuff, techniclal/build guides, nutrition, general encyclopaedia stuff. Again, most of it is blogs, podcasts, journals. I don't think it's unrealistic (in terms of resources) to build and maintain a bounded index of all of this and share signed versions with peers in Nostro/Vostro directory structure on a public key addressed home server. There could be an economic incentive for sharing perhaps?
This obvsly won't compete with huge centralised search engines, but I think it's probably true that after a certain volume of content, there's a diminishing return (in terms of value) the more that's indexed.
Appreciate your thoughts.
The web once was mostly text, and it still had only handful then only one search engine... The web is huge you can't crawl it cheaply even if it was only text.
Yes someone can do the indexing and sign it and share it, but who cares, how is that different from visiting a website, the index remains centralized and thus the Indexer will both comply with censorship AND look for ways to make money to offset the cost... That is also ignoring personalisation and suggestions entirely... Just a dumb index.
Again, someone has to download the data to discover it ... And that can't be made cheap
Best possible option is to let millions of people crawl small parts of the web, index them and generate a ZK proof of that and get paid for their work _somehow_ then share these parts with each other in some sort of a DHT.
However, a centralized indexer will always be 1000000x efficient because zk proofs are expensive as fuck.
Even then I don't think this will work well at all.
It is much easier and cheaper to just pay someone for a high quality aggregated feed
