You should be able to restore your follow list on https://nostrgram.co then visit your profile, click the follows button, and restore it.

As for why it happens, it sounds like it's a misbehaving relay or client that isn't sending your full contact list to all fo your relays.