GPS is passive, so if no nefarious app or the OS is not uploading location data, it's fine.

Without airplane mode, yes, of course there will be permanent records of your location stored.

GrapheneOS + open-source apps only + airplane mode is a pretty good combo.