The session is negotiated and established via an encrypted (giftwrapped) dead drop on any relay I decide to use. Relays give you the ability to do dead drops, totally unique on #nostr.
If the dead drop is cracked, the only info compromised is the session nonce which is useless after the channel session is closed. The only (very small) concern I have, is the potential harvest and decrypt threats for NIP44 encrypted payloads. Looking into post quantum algorithms for that.
Bottom line: don't need post quantum for ephemeral stuff, maybe for longer term stuff.