Who you follow *is* generally accurate because it is contained in 1 list which is easy for a client to retrieve. Your followers has to be pulled from every other user 1 at a time, from multiple relays, which is way less accurate.