The followers are collected from relays where others may published following you. So the count depends on which relays your current client is connected to and what relays your follower used to publish their contact list.
Discussion
Pfew. I’m not a developer or anything myself. I kinda understand it, but wow, looks complicated