Right, but it becomes harder to query those events, since you don't know who might be contacting you. ArcadeCity did something like this, but it only works for expected DMs (which could be considered a feature). The new gift wrap approach uses wrapping with ephemeral keys to prevent correlation while keeping addressing intact. So it's known that someone received a message, but not who sent it. Timing and padding attacks are also mitigated by timestamp randomization within a range and padding in the ciphertext. Relay fingerprinting is harder to avoid, but if you use a trusted relay or a proxy that can be mitigated as well.