The answer to my question seems to be something called "enclaves".
https://blog.opensecret.cloud/opensecret-technicals/
Enclaves seem to be something like a Virtual Machine that has its memory and resources protected from prying eyes at the hardware level, even from the hypervisor running the VM.
However, even if you offer reproducible builds of these enclaves, how can anyone be certain that the current running enclave really is that same build?
The enclave attestations include Amazon's AWS credentials. Ultimately the final trust point lies with AWS to honor the encryption for all of their secure enclave users.
Thoughts on Venice.ai?
Venice doesn’t use enclaves so we can’t see the code running on their servers. It follows a “trust we don’t log your chats” model, similar to VPN servers that ask you to trust they don’t log
Their service works well, though. You can pick your trust level. Maple is open source and provides cryptographic proof of the code running on the servers. Every user can verify that we don’t log the chats elsewhere.
