Some clients do this automatically, they try to find out which relays are used by the ones you follow and connect to them read-only. Then you don’t have to add them manually!

Otherwise relays would become more and more centralised over time, if everyone is forced to use the same relays.