Sometimes yes, but the relay is only visible on the profile page. The app has a chance to delete the cache.

Profile pictures keep showing up in multiple places at random. Caching is hard.