So you’re suggesting that the more money you have, the less time you’ll have to spend with your family? Isn’t the point of making more money to have a better life for you and your family?
Discussion
I don't think it's the money itself, but the responsibility that comes with it. To make that much money, you're typically very high up in the hierarchy, which comes with many people leaning on you.
I run a small business, and the responsibility that comes with it does take time away from my family, at times. I couldn't imagine what comes with leading 100-1,000 people.
There are very wealthy people that work 20 hours a week. Not all billionaires CEOs are working as hard as musk portrays himself to.
The top of one hierarchy is the bottom of another. 
explain.
I have a business, I am "at the top" of that hierarchy... which just means that I have to answer to clients... I'm at the bottom of the hierarchy that my clients are in.
“Working for yourself” really just means having 12 bosses instead of one. Being an entrepreneur means answering to the board. Being a venture capitalist means getting returns for your investors.
You are always accountable to someone. Just love what you do, do it well, and be a good human.
That is not what I'm suggesting. I'm suggesting that one does not accidentally become a billionaire. You have to make a lot of sacrifices to do that. Time with your children is almost certainly going to have to be one of them as kids are time consuming.
If you have a decent amount of money, that can help you have more free time... kinda... depending. But "billionaire"? Nah, that is an obsession.