I might be coming around to this position soon. I'm going to do some experimentation with supporting private groups with encryption vs AUTH'd relays.
The one thing that needs to be solved (if we assume relays can protect content) is leakage of content to the wider network through re-publishing. Gift wraps do this, but require decryption. Here's my idea for a lighter-weight method, I'm curious what you think of it:
When publishing a "private event", the signature is calculated based on the normal hash + the relay's url. But the relay's url isn't included in the event. This way, even if an event gets re-published to other relays, the signature will fail naive validation and be rejected. Relays could brute-force the signature by attempting validation using every known url, but this would have to be re-done every time the event was validated.
So, it basically works the same way paywalls do, not by making it impossible to share private content, but by making it too annoying to bother with.