The other consideration here is, maybe you are seeing poor Tor network connection quality.

You could always pause your node, turn off Tor, and finish your initial block download over a clear network.