Start with IPs in a replaceable event instead of fixed urls on each signed event. That's already a huge step.
Discussion
Can't do https with raw ips normally
Add a fragment identifier to the relay urls containing the cert fingerprint? Like this: "wss://69.69.69.69/endpoint #fp =
Pretty sure you can trust the event for the cert fingerprint, if it's wrong you'll just fail to reply or fetch related events.
In fact, does TLS between client and relay really just amount to MITM protection for privacy+censorship resistance?
I know that you like IPs, but DNS has nice features too, like (geo) balancing.
I repeat myself, but I find the whole process of IP assignment as much bigger problem. It's kindof central by design ...