I'll be honest, I'm not totally sure. Newer blocks are going to be heavier, just because they have more transactions in them.
Discussion
It's a bandwith issue. Just check your CPU usage while the wallet catches up, you'll see that's not the issue.
Nope, it's definitely not because if I synchronize it over the clear net, it's super fast. But synchronizing it over the hidden service is not. So that tells me it's not a CPU issue. It's definitely bandwidth.
Latest blocks are “a lot heavier” because not all nodes online have synchronized that blocks. So you will have a lower number of sources. Isn’t this the point?
The point was that it takes a lot longer to synchronize blocks using a hidden service than it does over the clear net. I knew it would take longer, but I did not realize how much longer it would take.