I assume you are using good relays. Most users have accepted whatever their first client has offered and many clients used to randomize relays from nostr.watch to try to decentralize the network from the bigger ones. When people don't know which relays they use, things can be quite slow.