If you go that route you are probably better off just running Bitcoin core with nothing on top.
Who do mean by "we"?
With proper opsec habits you should be fine.
Hardware wallets are beneficial for new people because as long as they can read and follow simple backup instructions the worst they can do is dox themselveves. It removes a lot of potential for human error.
(Not all hardware wallets are worth a dam. I like coldcard and jade.)