If you're on mobile and using most clients (which pull from all relays) it will save bandwidth and battery life to reduce the number of relays. You just need to be sure to keep the ones that the people you're following are using.

Or you could use a client like NostrGram that using a caching server to gather all the notes from 150+ top relays and send only a single stream to your device (no more pulling from a dozen or more relays) while still writing to all of your configured relays as per usual.

(Full disclosure: I'm the dev for NostrGram lol)