There would be no need for "trust" if Signal provided reproducible builds, because you could verify for yourself that the source code was not tampered with during the build process.
Discussion
It is not a would. One only needs to search for it and you find it:
https://github.com/signalapp/Signal-Android/tree/main/reproducible-builds
https://signal.org/blog/reproducible-android/
That's great.