I've been thinking about encrypted groups... The standard and model for them which is most flushed out is MLS which provides forward secrecy, double ratchet encrryption, and if a group admin rotates the keys post-compromise security. A similar protocol to Nostr, the p2panda protocol has gotten MLS working with just on log messages.
I think it's the right solution for encrypted private groups on Nostr.