It looks at Nostr content, not all of it, just stuff in the network of users of the Nos app and also all reports it can find across Nostr and checks that content and reports.

At the moment it takes them and runs them through openai’s moderation service, and if there’s a pretty high score, it creates a nostr report with labels for that content.

The Nos app uses it to generate content warnings for users. If you want to use it, you can simply follow nostr:npub14h23jzlyvumks4rvrz6ktk36dxfyru8qdf679k7q8uvxv0gm0vnsyqe2sh and if your client uses / displays reports from people you follow, you’ll see that there’s been a report and get a content warning on the content.

Unfortunately some nostr clients treat nostr report labels as replies and then people don’t understand how Nostr works and thinks somehow they’re getting reported to some shadowy organization which will remove their content from Nostr.

It is entirely opt-in. I think there needs to be many bots / services that could offer moderation if users wanted it.

https://platform.openai.com/docs/guides/moderation

https://github.com/planetary-social/cleanstr

There are two problems right now. One is the openai doesn’t always get things right, especially in cases like when people are talking about being harassed, it can’t figure out the context. The other issue is we’ve found that when a person creates a report on Nostr they’re often the target of harassment. So we’re building code that lets people send encrypted report events to a server where a person and the ai can both check, and if they agree, the reportinator can publish that report.

We need the two step process to both protect people making reports from being targeted, but also to have a check on reports, because both the ai and people can get things wrong.

Here’s the code for that:

https://github.com/planetary-social/reportinator_server

shi h zfgi ld?