Doesn't sound like a tool anyone could try out quickly :(
I wrote a director/producer/consumer setup
producer
connects to a bunch of relays, sends queries and pipes the results to a PUBSUB on redis (sharded by kind number)
director
orchestrates sending queries to the producers
consumers
does *something* with the data on the PUBSUB
(e.g. feeds nostr:npub1pages5yhmncmmfxuza9f9fx68fju0nxnewtlffzrapsufax5mv0sznuw9d, updates my neo4j, etc)
that's how I do it 🤷♂️ 😂
Discussion
oh yeah, definitely not; I wrote it months ago so now I have no idea how the hell it works anymore 😂