Maybe use Nostr to establish a multi-person channel, but use a secondary protocol to send messages more directly, like privately hosted or hidden relay services.

Encrypt messages with a shared private key, and encrypt the shared private key with each peer's public key, then transmit to each peer. Peer will encrypt/decrypt messages with the shared key. Peers may sign by own private to authenticate messages, allowing other peers to verify.

Generate new private keys upon membership shrink, or optionally upon any membership change for greater security (new peers can't read older messages).

For admin/moderation, a super key generates shared key(s).