I keep unfollowing people, for this reason. WoT doesn't work if someone just follows everyone back, indiscriminately.

If I unfollow, I still see their stuff (because they're usually followed by someone I still follow), but I don't see their follows, anymore, and it's a relief.