If you want to go this way, you essentially need to:

1. Create a root CA certificate

2. Create a server certificate, signed by the root CA

3. Install the server certificate in your docker container

4. Install the root CA certificate as trusted CA on any device that should be able to access the server (so that they accept your self-signed server certificate as valid)

There are a lot of details in this process that I didn't mention here. I really think it's a lot easier with a Letsencrypt certificate.