Counting follows is counting the number of contact lists you are on. Contact lists are large so relays will only allow you to pull so many at a time. To know how many people follow you is knowing every up to date contact list on every relay. Clients can’t really pull all of this information down, so depending which relays you are counting from you will get different numbers, but it will always be inaccurate. Follow counts are a centralized concept.