If you don't follow them to start with then you don't have to mute them. It's not like nostr if pushing content on you via an algorithm to decide what you do or do not want to see.
Furthermore, this is just noise. Nobody cares who you follow or don't follow. It's your nostr account to do with as you deem fit.