nostr:npub1ygrf5pf0ahkyyyrngv3twymqcz7pav6ssymrhdj90luscm9llszq56cg6z At no point did I actually think to look at their robots.txt… Honestly this is a good ledger of ‘known bad crawlers’ lol.

I found out years ago that (at that time anyways) they had a bunch of regular expressions saved off somewhere to match known spammy/URL shortner URLs so I grabbed those to use to filter my guestbook spam lol