I've had my eye on a library like that since April, just haven't had a chance to set it up. However long term I expect to be able to store images on nostr rather than my own server, so it will at that point become a problem for relay devs.
I believe that Microsoft offers a library that is an encrypted way to scan the images on your server for possible illegal photos and report them.
I think maybe Iris already uses it nostr:npub1g53mukxnjkcmr94fhryzkqutdz2ukq4ks0gvy5af25rgmwsl4ngq43drvk
There is a big market for third party safety tools like that to compete
Discussion
Is it like a suspicion or real evidence for that?
As I understand it, it usually is used as a flag for manual review
I think it cross matches it with real evidence that police have obtained. It might do other things too though
Update: it is called PhotoDNA. https://news.microsoft.com/features/microsofts-photodna-protecting-children-and-businesses-in-the-cloud/