Really interesting a sensitive topic!
In the last weeks I've been thinking to this very use case and imagining a "family & friends setup" that can bring kids both to a safe and enriching social experience, and private communication.
Raising new aware generations is essential to flourish the nostr ecosystem.
Yes
I've played with a test relay where use OpenAI moderation to filter out content. Same time also filter out content based on obvious keywords like nsfw. Seems to work well but need more on images side.
Wait you query openai on each event or how does it work?
Yes, was not what I was looking to do with our relays so did not pursue much. I think more potential in building custom model and use that free of OpenAI. Or find existing open models.
https://gitlab.com/soapbox-pub/strfry-policies/-/blob/develop/src/policies/openai-policy.ts
Did you measure the latency introduced?
No never got that far into it. I for sure don't like depending on 3rd party API. But I am gathering training data to use with a custom model from our relays and honeypot accounts.
YeH, for sure, running it locally would be the way to go; too expensive and too systolic feeding everything to Sam Cuckman
