If you're only connected to a small number or fast relays, then your client has less connections and less data to send out and receive.