Depends on what client you're using (amethyst, days, primal) and what relays you're connected to. Could be that you are still following me but not every relay has it etc.

Follows and followers are a little clunky on nostr for the time being