Removing a few relays shouldn't make me go from following 3.5K people, to 0. I still have 10 relays that I've been using this whole time.
It's not the first time this has happened, there's some kind of bug with relays that makes them overwrite your follows with a blank list when you start adjusting them.
The quality of interacting with the protocol varies between clients. I have started to prefer using https://metadata.nostr.com/ to adjust things like info and relays.
It has at least for me greatly reduced the loss of data. Plus the site easily let's you fo a backup of your meta data. Cheers & good luck!
