It's your connection followers recounts everytime you look again at a profile. You didn't lose it. You just might be in a area with bad signal & it's stalling to count.
Discussion
They're definitely gone. I just re-followed a few, and I can confirm by looking at my account from different devices. Looks like it's a race condition, but should be solved (or less common) when more clients implement NIP-65.