Yeah for the encryption, but ecdh gives you a shared secret you can use to derive uncorellated key pairs for the DM session.

ex:

tweak(my_private_key, shared_secret) = my_dm_private_key

since ECC is nice that way, my DM partner also knows:

tweak(my_public_key, shared_secret) = my_dm_public_key

You'd need to work to defeat timing attacks between this handshake and your first DMs, but the new key pairs are publicly uncorrellated from your "normal" key pair.