It is being stored by the app, because unless it is being stored by another application like nos2x or Amber, the client can, and in fact needs to get the nsec back so that it can sign and decode with it.

The app can't just reference it, unless the keychain does the signing and decryption, which I don't think it does, but maybe I'm wrong here?

It is just storing it in a different memory location. It doesn't protect you from the client being malicious.