Agreed it's better if the server is open source, but as long as the client is open source and created the private key to sign the messages and client side encrypts the messages, there's little need to worry other than the Metadata that goes with it. When the message goes out, one needs to assume any relay or packet sniffer will attempt to read it, including the Signal server. But ideally everything is open source so we can see what it's doing with the metadata rather than trusting it.