this proposal is more client-side, but it made me think of https://github.com/bluesky-social/jetstream which is a way of viewing all of the atproto network wayy more efficiently and you can also filter through it to choose which content you want it to search.