You can also start off with just a paper wallet for cold storage if you don't plan to transact day to day

Then as your stack increases in purchasing power you can grab a hardware device.

Also a tap signer might be a good choice too on a budget