Fetching all users followed by you would be a mess, just like, currently, fetching the list of people that follow you is a mess (and clients may get it wrong, by listing people that used to follow you but no longer do).

Remember that relays have different data and no relay has all data. So if you follow and then unfollow me, even if the specification says the unfollow event must delete the follow event, some relays will still have the follow event. The client would have to fetch all your follow and unfollow event and the list will be complete only once it has received all events.

Currently, the client still has to fetches multiple relays and wait for multiple answers, then pick the one with the most recent date, however that's less work than it would be required if there were follow events.

I still think that individual follow events (but not really unfollow events) should exist for the purpose of providing a notification when someone follows you.