If you used a software wallet, your private key has been on your computer, and hence was potentially exposed to a hacker. With a hardware wallet, the key never leaves the wallet itself.

A hardware wallet is also very useful for making transactions. How do you make a transaction with your private key on paper? You can't, you have to re-enter it into your computer (again exposing it to hackers). The hardware wallet allows you to sign a transaction while not exposing the private key.

Yes, you have to trust the hardware maker, but the good ones are as open/verifiable as possible.