Replying to Avatar Leo Wandersleb

What is the best tool to scrape all relays for specific events as a one-off task?

For example I have a list of 50 relays and want to know which version (event id) of my profile is stored on each of them.

My current approach is to go to https://nostr.info/relayr/ , patch the `ws.onmessage` callback to `console.log` what I need and then to either use the UI or further call on the dev console what the UI would call.

Fellow nostr devs, how do you do this?

I wrote a director/producer/consumer setup

producer

connects to a bunch of relays, sends queries and pipes the results to a PUBSUB on redis (sharded by kind number)

director

orchestrates sending queries to the producers

consumers

does *something* with the data on the PUBSUB

(e.g. feeds nostr:npub1pages5yhmncmmfxuza9f9fx68fju0nxnewtlffzrapsufax5mv0sznuw9d, updates my neo4j, etc)

that's how I do it 🤷‍♂️ 😂

Reply to this note

Please Login to reply.

Discussion

Doesn't sound like a tool anyone could try out quickly :(