The other consideration here is, maybe you are seeing poor Tor network connection quality.
You could always pause your node, turn off Tor, and finish your initial block download over a clear network.
Discussion
See the section about using connect=localip - I cheated the entire process by forcing Core to only use a synced node on my LAN via gigabit ethernet.