You can use Proton with your own domain, but the core Proton services are not designed for self-hosting.

While Proton is open source on the client side, their service infrastructure is proprietary and cannot be self-hosted on a NAS or any other personal hardware.

AFAIK, the only self-hosting related options with Proton are using a custom domain with Proton services and utilizing their SMTP bridge for integration with other applications.