That was kind of my experience running a Mastodon server myself. Because I didn't have any speech policy for the server that I ran for myself and two friends. I got blocked by a large portion of the entire network, meaning That the servers that would actually federate with me were all full of Nazis and assholes and pedophiles and basically anyone that the rest of the network didn't want to federate with for good reason.