The reason why signal is so popular is easy and reliable asynchronous comms. I don't believe you can achieve this with a p2p stack. At least with a signal-level ux.
Discussion
Could it be part of the protocol that relays only send encrypted messages if they know the person receiving is the intended recipient? In that sense the relay is behaving more like a private server and should solve any privacy concerns.