yeah, they are only reading, not writing, so all that would really be required is to slow them down

probably could split the direct followed vs follows of follows into two tiers also so that the second level of depth get less service quality

probably maybe should work on it today because this problem with scrapers is fairly annoying

it's only nostr WoT spiders at this point, i wish i could just suggest to the spider people to just open a subscription for these and catch all the new stuff live, that would be more efficient for them and less expensive for relay operators

at this point it's fairly early in the game for that stuff so maybe some education of WoT devs might help...

cc: nostr:npub176p7sup477k5738qhxx0hk2n0cty2k5je5uvalzvkvwmw4tltmeqw7vgup there is no point in doing sliding window spidering on events once you have the history, from then on just open a subscriptiion

this would be more friendly from the AI spiders if they had a way to just catch new updates, but unless dey pay me! i'm just gonna tarpit them