No, once it's in your wallet, it's there. Even if the relay disappears. Only payments you haven't received yet (while you were offline) can potentially disappear if all relays they are published to disappear.