That's probably it. I always assumed you could disable caching if you unselected "Enhanced Privacy" in the settings so that you connect to relays individually but I think that was wrong. The caching service seems to always stay connected. I'll just stop using it for the time being.