I'm sorry to hear, that's rough. I wouldn't completely discount the panic attack theory, it's possible, the flight or fight response is very powerful and subconscious things can trigger it. If you have sleep apnea then it could be causing a panic response, for example, and the inhaler could just be psychological. This is a good read: https://ok0.org/Fight-or-flight-response.doc

I also think you're right to be skeptical and should try to get more tests done, particularly to rule out issues with your heart. A blood test would be useful too and easy to do. Getting in front of an heart specialist can be expensive, there are some gadgets which can give you a basic EKG such as those by https://kardia.com/

Take all of this for what it is, which is the opinion of a random dude on the Internet who isn't a doctor. I hope you feel better.