That’s not what you said though. You said people were working more hours to get less. And it’s not clear to me it’s believable that the whole world could be literally 100 times wealthier than we are right now.
Discussion
It's hard to compare if we get less for our work or not as you can't compare a smartphone or a cure for a disease with not having those back then but I find the question very valid how can it be that 120 years after the very invention of automation (conveyor belt) most of us still have to labor most of our waking hours in order to support a family?
Average hours worked is also not up. Where are you getting your information from when you say things? 
Maybe that's women entering the work force? While the prior generation mostly had single providers, more and more women entering the work force often part time resulted in statistical work hour reductions as in your chart.
https://www.epi.org/publication/webfeatures_snapshots_07072004/
I would just suggest you're working really hard to engage in narrative-fitting, here.
Well, I did ask myself this question since early in school: If GDP grows some percent per year since 100 years, why is it most of us still have to spend most of our time working? Are we really 100 times better off today? Why can't I just work 5h/week and be 20 times better off than back then?