That's probably it. I always assumed you could disable caching if you unselected "Enhanced Privacy" in the settings so that you connect to relays individually but I think that was wrong. The caching service seems to always stay connected. I'll just stop using it for the time being.
Discussion
I believe they also do very aggressive caching on THEIR servers. as in, a layer between you and the relays. perhaps a *way too thick* layer