I'm not sure how you should prioritize your time, but I would suggest a sort of "baseline freedom algorithm" for any LLM filtering, something like this:

The user has a toggle at the top of the UI to flip between soft filter and hard filter. A link nearby explains the details.

The soft filter and hard filter also change the color scheme of the UI, at least slightly. This is really useful, not just cosmetic - screenshot context clarity, etc.

If at least 7 of the last 10 posts are currently questions, the toggle defaults to soft filter for all users. If something is flagged as a non-question, it gets a red or yellow box around it saying it might not be a question, and the site won't accept any more non-question from that npub for 12 hours. They get an auto-reply explaining this. If they submit another non-question after their 12 hour cooldown, it goes through again, but their new cooldown doubles to 24 hours, then 48 if it happens again, etc.

If the site has 2 or 3 non-question posts in a row to display, then instead of showing them fully, the user just sees a little box saying "x number of posts might not be questions, click here to display them"

If at least 4 of 10 recent posts on display are non-questions, then the user gets a warning saying "something is wrong, hard filtering enabled by default" and unless the user toggles back to the soft filter, the site just skips loading anything flagged as a non-question.