Better to do it sooner rather than later. You could end up with followers on all those relays. Could lose them if you prune the list and they are not on another common relay. You’d break the link to them.