Yeah for the encryption, but ecdh gives you a shared secret you can use to derive uncorellated key pairs for the DM session.
ex:
tweak(my_private_key, shared_secret) = my_dm_private_key
since ECC is nice that way, my DM partner also knows:
tweak(my_public_key, shared_secret) = my_dm_public_key
You'd need to work to defeat timing attacks between this handshake and your first DMs, but the new key pairs are publicly uncorrellated from your "normal" key pair.