Don't use their software, then there is nothing to worry about. The hardware itself is not bad.
Discussion
It has closed source. How do you know it's not bad?
No it isn't closed 🤗 You can check all the details here: https://support.ledger.com/hc/en-us/articles/11132311094813-Ledger-s-open-source-roadmap
Except the parts of the firmware on the device, due to an NDA with the manufacturer of the secured element chip.
How do you know there's zero backdoors in the device?
Maybe that's a good thing?
There are already many arguments and discussions about that part and to date, there is no known case where this has led to a problem. Easy✌️
So you don't know if there are any backdoors but you decided to trust the company.
Good luck with that 🤙💜