What client are you using? For basic inbox/outbox setup, 4 is plenty. Some clients, like #Amethyst, have far more robust relay settings for a lot more control over which relays are used for what purpose.