Technically speaking someone could be following you but not have any relays in common with you and send you a DM and you'd never get it because you're not using the same relays. This is why the new outbox model is very important. Many clients are moving to this new method. Primal uses a caching service on their backend which also helps here.