Yeah that happens. There are actually tools to do exactly this with video, but I haven't played with it for months. They help with keeping things consistent between frames. I'll see if I can find a link.