The only certainty you can have is to audit the code yourself and spot a vulnerability. There is a degree of trust required with all hardware signing devices. If the firmware update isn't enough to rebuild your eroded trust, might need to consider an alternative.

You can certainly keep the device and user for a smaller "spending" wallet or include it in a multi-sig. Shoot you could just use it as your 2FA device for signing in.