Go to system settings, developer options and check running services, your gboard is running extra services even when it's without permissions.

I had similar setup with it, but I also had Google play services installed. The Gboard app communicated with google play services without any permissions. If you have installed Google play services and it is rubbing alongside Gboard, it is not such a sure thing that what you type stays on your device.