Nostr Web Client

uncomfortable truth? : i feel like the debate on the mailing list about AI authorship for BIPs is reminiscent of the filtering debate for Bitcoin. In both cases, people arguing for prevention (not just dissuasion) are essentially pissing in the wind. Using "automated tools to detect AI usage in documents" is doomed to fail over time, with terrible false +ve rates; just think about it. In both cases, the technological reality just does not support the prevention argument.

As for dissuasion, even that's problematic. It's hard to argue these tools are not useful, and the nuance of "don't misuse it" is too difficult to capture imo. In short, just like spam on-chain, I'm not saying there isn't a problem, but it's hard to define and the cure is worse than the disease.

Reply to this note

Please Login to reply.

Discussion

aj 1mo ago

The point is to allow editors to reject ai "slop" without having to treat it as if it were a serious contribution, and without getting blamed for being overly controlling, unpredictable or arbitrary.

Using the same approach as for onchain spam -- only N prs will be accepted per week and whoever pays the editors the most will have their prs prioritised -- probably doesn't have the right incentives.

waxwing 1mo ago

I don't think any reviewer should have a responsibility to justify in detail why they're not accepting a PR, or why they're choosing to spend more time to review one and not another.

I mean, heck, it's going to happen right - a PR from sipa vs a PR from Joe Bloggs.

I think your point about the analogy isn't as clear as all that; the analogy would be more with setting policy and less with what constitutes a valid transaction (that's more, does the syntax of the document even satisfy the definition of a BIP).

If the problem with AI drafts were *only* volume we wouldn't be having the same discussion, nor using terms like slop.

aj 1mo ago

You might not think bip editors have that responsibility, but lots of people do, and will react pretty negatively if they don't appear to live up to that responsibility.

fiatjaf 1mo ago

People need a way to filter out stuff, because seriously considering every crazy idea ever thought by anyone isn't a feasible option.

Before, the act of writing a BIP was proof-of-work enough to pass the filter and be worth reading by someone else, now that isn't true anymore, so new filters have to be found. For now, just filtering on what is written by AI or not is not great but I can't think of anything better.

Disclaimer: I have no idea of how the work on BIPs is done or what is the latest drama.

waxwing 1mo ago

Filtering is inevitable. I fear having a fixed rule about how to filter (such as an AI filtering tool) may not be helpful. It's certainly no that clear-cut, though.