Subnostr

Hi praxeologist, good question. We use an AI model designed to flag and report CSAM. If it is not 90% sure, we will manually review the media. CSAM is removed and reported to the NCMEC. The account is then blocked from using our service.

We also filter media that suggests anything inappropriate with a child including cartoons, adult clothing, positions, etc. We have recently stopped allowing free adult porn (intercourse) uploads.

We have experimented with the Cloudflare filter and Microsoft PhotoDNA, but find the AI model is most accurate.

More info can be found here:

https://nostr.build/features/

praxeologist 7mo ago

Thank you for the information. I have a few follow-up questions regarding the AI model you mentioned.

Was the model developed in-house, or is it based on existing models?

If it is based on existing models, could you specify which ones were used?

Can you provide more details on the training process and the data used to train the model?

I appreciate your help in understanding this better!

Reply to this note

Please Login to reply.

Discussion

nostr.build 7mo ago

All of our content is preliminary scanned with the Cloudflare CSAM api, but it doesn’t catch everything..

The AI model was existing on Hugging Face.

Adding nostr:npub137c5pd8gmhhe0njtsgwjgunc5xjr2vmzvglkgqs5sjeh972gqqxqjak37w for more details.

Do you have feedback, issues or recommendations?

https://developers.cloudflare.com/cache/reference/csam-scanning/ CSAM Scanning Tool · Cloudflare Cache (CDN) docs

https://huggingface.co/mbehbooei/vit-base-patch16-224-in21k-finetuned-moderation

praxeologist 7mo ago

Thanks it helps a lot! I was developing an open source blossom server and I was conducting research on CSAM filtering. that's why I asked you guys for help.

btw here's the what I've done so far:

https://github.com/dezh-tech/nos3/